测试人员和敏捷团队实用指南
A Practical Guide for Testers and Agile Teams
新泽西州上萨德尔河 • 波士顿 • 印第安纳波利斯 • 旧金山
纽约 • 多伦多 • 蒙特利尔 • 伦敦 • 慕尼黑 • 巴黎 • 马德里
开普敦 • 悉尼 • 东京 • 新加坡 • 墨西哥城
Upper Saddle River, NJ • Boston • Indianapolis • San Francisco
New York • Toronto • Montreal • London • Munich • Paris • Madrid
Capetown • Sydney • Tokyo • Singapore • Mexico City
制造商和销售商用来区分其产品的许多名称都被声明为商标。如果这些名称出现在本书中,并且出版商知道商标声明,则这些名称的首字母大写或全部大写。
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals.
作者和出版商精心准备了本书,但不作任何明示或暗示的保证,也不对错误或遗漏承担任何责任。对于与使用此处包含的信息或程序有关或因使用此处包含的信息或程序而引起的附带或间接损害不承担任何责任。
The authors and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein.
为批量购买或特价销售而订购大量本书时,出版商会提供极好的折扣,其中可能包括电子版和/或定制封面以及适合您的业务、培训目标、营销重点和品牌兴趣的内容。获取更多资讯,请联系:
The publisher offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales, which may include electronic versions and/or custom covers and content particular to your business, training goals, marketing focus, and branding interests. For more information, please contact:
美国公司和政府销售
(800) 382-3419
corpsales@pearsontechgroup.com
U.S. Corporate and Government Sales
(800) 382-3419
corpsales@pearsontechgroup.com
对于美国以外的销售,请联系:
For sales outside the United States, please contact:
国际销售
international@pearson.com
International Sales
international@pearson.com
请访问我们的网站:informit.com/aw
Visit us on the Web: informit.com/aw
美国国会图书馆在编目数据:
Library of Congress Cataloging-in-Publication Data:
克里斯平,丽莎。
敏捷测试:测试人员和敏捷团队实用指南 /
Lisa Crispin,Janet Gregory。— 第一版。
p. 厘米。
包括参考书目和索引。
ISBN-13: 978-0-321-53446-0 (pbk. : alk. paper)
ISBN-10: 0-321-53446-8 (pbk. : alk. paper) 1. 计算机软件——
测试。2. 敏捷软件开发。I. 格雷戈里,珍妮特。二。标题。
QA76.76.T48C75 2009
005.1—dc22
2008042444
Crispin, Lisa.
Agile testing : a practical guide for testers and agile teams /
Lisa Crispin, Janet Gregory. — 1st ed.
p. cm.
Includes bibliographical references and index.
ISBN-13: 978-0-321-53446-0 (pbk. : alk. paper)
ISBN-10: 0-321-53446-8 (pbk. : alk. paper) 1. Computer software—
Testing. 2. Agile software development. I. Gregory, Janet. II. Title.
QA76.76.T48C75 2009
005.1—dc22
2008042444
版权所有 © 2009 Pearson Education, Inc.
Copyright © 2009 Pearson Education, Inc.
版权所有。在美利坚合众国印刷。本出版物受版权保护,在任何禁止的复制、存储在检索系统中或以任何形式或通过任何方式(电子、机械、影印、录音或类似方式)传输之前,必须获得出版商的许可。要获得使用本作品材料的许可,请向 Pearson Education, Inc., Permissions Department, 200 Old Tappan Road, Old Tappan, New Jersey 07675 提交书面请求,或者您可以将请求传真至 (201) 236-3290 .
All rights reserved. Printed in the United States of America. This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise. To obtain permission to use material from this work, please submit a written request to Pearson Education, Inc., Permissions Department, 200 Old Tappan Road, Old Tappan, New Jersey 07675, or you may fax your request to (201) 236-3290.
致我的丈夫鲍勃唐宁——你是蜜蜂的膝盖!
——丽莎
To my husband, Bob Downing—you’re the bee’s knees!
—Lisa
致 Jack、Dana 和 Susan,以及我家族中的所有作家。
——珍妮特
To Jack, Dana, and Susan, and to all the writers in my family.
—Janet
还有我们最喜欢的驴和龙。
——丽莎和珍妮特
And to all our favorite donkeys and dragons.
—Lisa and Janet
Chapter 1. What Is Agile Testing, Anyway?
What Do We Mean by “Agile Testing”?
A Little Context for Roles and Activities on an Agile Team
Interaction between Customer and Developer Teams
How Is Agile Testing Different?
Chapter 2. Ten Principles for Agile Testers
Applying Agile Principles and Values
Enable Face-to-Face Communication
Practice Continuous Improvement
Part II. Organizational Challenges
Chapter 3. Cultural Challenges
Barriers to Successful Agile Adoption by Test/QA Teams
Not Understanding Agile Concepts
Cultural Differences among Roles
Speaking the Manager’s Language
Work On Your Own Professional Development
Beware the Quality Police Mentality
Integration of Testers into an Agile Project
Every Team Member Has Equal Value
Chapter 5. Transitioning Typical Processes
Why Should We Use a Defect Tracking System (DTS)?
Test Strategy vs. Test Planning
Frameworks, Models, and Standards
Part III. The Agile Testing Quadrants
Chapter 6. The Purpose of Testing
Tests that Critique the Product
Chapter 7. Technology-Facing Tests that Support the Team
The Purpose of Quadrant 1 Tests
Why Write and Execute These Tests?
Designing with Testing in Mind
Where Do Technology-Facing Tests Stop?
What If the Team Doesn’t Do These Tests?
Chapter 8. Business-Facing Tests that Support the Team
Driving Development with Business-Facing Tests
Chapter 9. Toolkit for Business-Facing Tests that Support the Team
Business-Facing Test Tool Strategy
Tools to Elicit Examples and Requirements
Tools for Automating Tests Based on Examples
Tools to Test below the GUI and API Level
Tools for Testing through the GUI
Use Appropriate Test Design Patterns
Automated vs. Manual Quadrant 2 Tests
Chapter 10. Business-Facing Tests that Critique the Product
Automation and Exploratory Testing
User Needs and Persona Testing
Testing Documents and Documentation
Tools to Assist with Exploratory Testing
Chapter 11. Critiquing the Product Using Technology-Facing Tests
Performance, Load, Stress, and Scalability Testing
Performance and Load-Testing Tools
Chapter 12. Summary of Testing Quadrants
Review of the Testing Quadrants
The Automated Functional Test Structure
Critiquing the Product with Business-Facing Tests
Using the Agile Testing Quadrants
Chapter 13. Why We Want to Automate Tests and What Holds Us Back
Manual Processes Are Error Prone
Automation Frees People to Do Their Best Work
Automated Regression Tests Provide a Safety Net
Automated Tests Give Feedback, Early and Often
Tests and Examples that Drive Coding Can Do More
Barriers to Automation—Things that Get in the Way
Programmers’ Attitude—“Why Automate?”
The “Hump of Pain” (The Learning Curve)
Can We Overcome These Barriers?
Chapter 14. An Agile Test Automation Strategy
An Agile Approach to Test Automation
Continuous Integration, Builds, and Deploys
What Might Be Hard to Automate?
Developing an Automation Strategy—Where Do We Start?
Think about Test Design and Maintenance
Applying Agile Principles to Test Automation
Taking the Time to Do It Right
Apply Agile Coding Practices to Tests
When Database Access Is Unavoidable or Even Desirable
Identifying Requirements for Your Automation Tool
Part V. An Iteration in the Life of a Tester
Chapter 15. Tester Activities in Release or Theme Planning
The Purpose of Release Planning
The Tester’s Role in Sizing Stories
Testing Considerations While Prioritizing
Tracking Test Tasks and Status
Chapter 16. Hit the Ground Running
Potential Downsides to Advance Preparation
Customers Speak with One Voice
Geographically Dispersed Teams
Chapter 18. Coding and Testing
Coding and Testing Progress Together
Tests that Critique the Product
Is It a Defect or Is It a Feature?
Choose the Media You Should Use to Log a Bug
Alternatives and Suggestions for Dealing with Bugs
Testers Facilitate Communication
Chapter 19. Wrap Up the Iteration
Chapter 20. Successful Delivery
Planning Enough Time for Testing
Integration with External Applications
Data Conversion and Database Updates
Post-Development Testing Cycles
Chapter 21. Key Success Factors
Success Factor 1: Use the Whole-Team Approach
Success Factor 2: Adopt an Agile Testing Mind-Set
Success Factor 3: Automate Regression Testing
Success Factor 4: Provide and Obtain Feedback
Success Factor 5: Build a Foundation of Core Practices
Coding and Testing Are Part of One Process
Success Factor 6: Collaborate with Customers
“随着敏捷方法进入主流,我们已经了解了很多关于测试规程如何适应敏捷项目的知识。Lisa 和 Janet 让我们清楚地了解在敏捷测试中应该做什么,应该避免什么。”
“As Agile methods have entered the mainstream, we’ve learned a lot about how the testing discipline fits into Agile projects. Lisa and Janet give us a solid look at what to do, and what to avoid, in Agile testing.”
—Ron Jeffries,www.XProgramming.com
—Ron Jeffries, www.XProgramming.com
“对敏捷及其如何影响软件测试社区的精彩介绍!”
“An excellent introduction to agile and how it affects the software test community!”
—Gerard Meszaros,敏捷指导和精益软件开发咨询公司 Solution Frameworks, Inc. 的敏捷实践主管和首席测试策略师
—Gerard Meszaros, Agile Practice Lead and Chief Test Strategist at Solution Frameworks, Inc., an agile coaching and lean software development consultancy
“在体育和音乐领域,人们知道练习技巧的重要性,直到它成为他们做事方式的一部分。本书介绍了软件开发中一些最基本的技术——如何将质量构建到代码中——这些技术应该成为每个开发团队的第二天性。这本书广泛而深入地介绍了如何将测试移至开发过程的前端,并大量介绍了使本书栩栩如生的现实示例。”
“In sports and music, people know the importance of practicing technique until it becomes a part of the way they do things. This book is about some of the most fundamental techniques in software development—how to build quality into code—techniques that should become second nature to every development team. The book provides both broad and in-depth coverage of how to move testing to the front of the development process, along with a liberal sprinkling of real-life examples that bring the book to life.”
—Mary Poppendieck,《精益软件开发》和《实施精益软件开发》一书的作者
—Mary Poppendieck, Author of Lean Software Development and Implementing Lean Software Development
“令人耳目一新的实用主义。充满智慧。没有教条。这本书改变了游戏规则。每个软件专业人士都应该阅读它。”
“Refreshingly pragmatic. Chock-full of wisdom. Absent of dogma. This book is a game-changer. Every software professional should read it.”
——Bob Martin 叔叔,Object Mentor, Inc.
—Uncle Bob Martin, Object Mentor, Inc.
“通过敏捷测试,Lisa 和 Janet 使用他们对测试的整体敏感性来描述愿意提高测试效率的测试人员和团队的文化转变。现实生活中的项目经验和特定技术的结合提供了一种学习和适应不断变化的项目需求的绝佳方式。”
“With Agile Testing, Lisa and Janet have used their holistic sensibility of testing to describe a culture shift for testers and teams willing to elevate their test effectiveness. The combination of real-life project experiences and specific techniques provide an excellent way to learn and adapt to continually changing project needs.”
—Adam Geras,理学硕士 开发测试人员,Ideaca 知识服务
—Adam Geras, M.Sc. Developer-Tester, Ideaca Knowledge Services
“在敏捷项目中,每个人似乎都在问,'但是,测试呢?' 这完全是开发团队、测试团队的责任,还是开发人员和测试人员之间的协作?或者,“我们应该自动化多少测试?” 丽莎和珍妮特写了一本书,终于回答了这些类型的问题,甚至更多!无论您是测试人员、开发人员还是经理,您都将从他们在这本出色的书中分享的真实工作经验中学到许多很好的例子和故事。”
“On Agile projects, everyone seems to ask, ‘But, what about testing?’ Is it the development team’s responsibility entirely, the testing team, or a collaborative effort between developers and testers? Or, ‘How much testing should we automate?’ Lisa and Janet have written a book that finally answers these types of questions and more! Whether you’re a tester, developer, or manager, you’ll learn many great examples and stories from the real-world work experiences they’ve shared in this excellent book.”
—Paul Duvall,Stelligent 的首席技术官和《持续集成:提高软件质量和降低风险》的合著者
—Paul Duvall, CTO of Stelligent and co-author of Continuous Integration: Improving Software Quality and Reducing Risk
“终于出版了一本面向敏捷团队测试人员的书,它承认不只有一种正确的方法!敏捷测试全面覆盖了测试人员在转向敏捷时面临的问题:从工具和指标到角色和流程。它以来自许多贡献者的大量故事和示例进行了说明,清楚地描述了当今成功的敏捷测试人员正在做什么。”
“Finally a book for testers on Agile teams that acknowledges there is not just one right way! Agile Testing provides comprehensive coverage of the issues testers face when they move to Agile: from tools and metrics to roles and process. Illustrated with numerous stories and examples from many contributors, it gives a clear picture of what successful Agile testers are doing today.”
—WatirCraft 首席技术官兼 Watir 首席开发人员 Bret Pettichord
—Bret Pettichord, Chief Technical Officer of WatirCraft and Lead Developer of Watir
“质量是内在的,”程序员不断告诉我。作为拟议收购的一部分,我的老板要求我对开发团队及其产品进行最后的尽职调查。我们已经确定该公司最近推出的产品在市场上表现良好,但我要确保我们不会买到比收益更多的麻烦。所以我把时间都花在了开发团队身上。我一直在寻找可能因匆忙发布产品而引起的问题。我想知道,“代码干净吗?是否存在只能由一名开发人员处理的模块?是否有成百上千的缺陷等待被发现?” 当我问及团队的测试方法时,“质量是内在的”是我得到的答案。
“Quality is baked in,” the programmers kept telling me. As part of a proposed acquisition, my boss had asked me to perform some final due diligence on the development team and its product. We’d already established that the company’s recently launched product was doing well in the market, but I was to make sure we were not about to buy more trouble than benefit. So I spent my time with the development team. I was looking for problems that might arise from having rushed the product into release. I wondered, “Was the code clean? Were there modules that could only be worked on by one developer? Were there hundreds or thousands of defects waiting to be discovered?” And when I asked about the team’s approach to testing, “Quality is baked in” was the answer I got.
因为这种相当不寻常的口语化可能意味着任何事情,所以我进一步追问。我发现这是公司创始人表达质量先驱 W. Edwards Deming 著名的十四点之一的速记:将质量融入产品而不是在以后尝试测试它。
Because this rather unusual colloquialism could have meant just about anything, I pressed further. What I found was that this was the company founder’s shorthand for expressing one of quality pioneer W. Edwards Deming’s famous fourteen points: Build quality into the product rather than trying to test it in later.
将质量构建到他们的产品中的想法是敏捷团队工作方式的核心。敏捷团队在短期迭代中工作,部分是为了确保应用程序保持已知的质量状态。敏捷团队是高度跨职能的,程序员、测试人员和其他人在每次迭代中并肩工作,这样质量就可以通过验收测试驱动开发、高度重视自动化测试和整体-团队思维。优秀的敏捷团队通过不断构建他们的产品,在新工作完成后几分钟内整合新工作,从而融入质量。敏捷团队利用重构等技术和对简单性的偏好来防止技术债务的积累。
The idea of building quality into their products is at the heart of how agile teams work. Agile teams work in short iterations in part to ensure that the application remains at a known state of quality. Agile teams are highly cross-functional, with programmers, testers, and others working side by side throughout each iteration so that quality can be baked into products through techniques such as acceptance-test driven development, a heavy emphasis on automated testing, and whole-team thinking. Good agile teams bake quality in by building their products continuously, integrating new work within minutes of its being completed. Agile teams utilize techniques such as refactoring and a preference for simplicity in order to prevent technical debt from accumulating.
学习如何做这些事情是很困难的,对于测试人员来说尤其如此,他们的角色在以前的书中很少受到关注。幸运的是,您现在手里拿着的这本书回答了每个开始从事敏捷项目的测试人员心中的问题,例如:
Learning how to do these things is difficult, and especially so for testers, whose role has been given scant attention in previous books. Fortunately, the book you now hold in your hands answers questions on the mind of every tester who’s beginning to work on an agile project, such as:
我的角色和职责是什么?
What are my roles and responsibilities?
我如何与程序员更紧密地合作?
How do I work more closely with programmers?
我们自动化了多少,我们如何开始自动化?
How much do we automate, and how do we start automating?
丽莎和珍妮特的经历在书中的每一页都闪闪发光。然而,这本书不仅仅是他们的故事。在本书中,他们结合了来自真实世界的敏捷测试人员的数十个故事。这些故事构成了这本书的核心,也是它如此独特的原因。在象牙塔中大喊“这是进行敏捷测试的方法”是一回事。讲述团队的故事是另一回事,这些团队在可用性测试、抵制自动化的遗留代码、习惯于传统阶段门开发的过渡测试人员、“跟上”短迭代的测试等挑战中变得敏捷并取得胜利,并知道某个功能何时“完成”。
The experience of Lisa and Janet shines through on every page of the book. However, this book is not just their story. Within this book, they incorporate dozens of stories from real-world agile testers. These stories form the heart of the book and are what makes it so unique. It’s one thing to shout from the ivory tower, “Here’s how to do agile testing.” It’s another to tell the stories of the teams that have struggled and then emerged agile and victorious over challenges such as usability testing, legacy code that resists automation, transitioning testers used to traditional phase-gate development, testing that “keeps up” with short iterations, and knowing when a feature is “done.”
Lisa 和 Janet 一开始就在那里,学习如何进行敏捷测试,当时普遍认为敏捷团队不需要测试人员,程序员可以自己提高质量。多年来,通过文章、会议演示以及与他们的客户和团队的合作,Lisa 和 Janet 帮助我们看到了测试人员在敏捷项目中扮演的丰富角色。在这本书中,Lisa 和 Janet 使用测试自动化金字塔、Brian Marick(他自己是另一位世界级的敏捷测试员)的敏捷测试象限,以及其他技术来展示在认为测试是必要的但不可行的心态中缺失了多少测试人员不是。
Lisa and Janet were there at the beginning, learning how to do agile testing back when the prevailing wisdom was that agile teams didn’t need testers and that programmers could bake quality in by themselves. Over the years and through articles, conference presentations, and working with their clients and teams, Lisa and Janet have helped us see the rich role to be filled by testers on agile projects. In this book, Lisa and Janet use a test automation pyramid, the agile testing quadrants of Brian Marick (himself another world-class agile tester), and other techniques to show how much was missing from a mind-set that said testing is necessary but testers aren’t.
如果你想学习如何将质量融入你的产品,或者是一个有抱负的敏捷测试人员,想了解你的角色,我认为没有比 Lisa 和 Janet 更好的指南了。
If you want to learn how to bake quality into your products or are an aspiring agile tester seeking to understand your role, I can think of no better guides than Lisa and Janet.
想象一下,你自己掠过几千年前的风景,看着下面的人。他们在充满敌意的领土上勉强维持生计,打猎、捕鱼和少量种植。在远处,您会看到冰川的闪光。走近一点,你会发现它正在快速融化,几乎要筑成一个巨大的湖泊。当您观看时,湖水破土而出,顺着河床冲下,将河床冲刷得更深,溅起水花溅到远处景观的悬崖上——其中一些已经坍塌。
Imagine yourself skimming over a landscape thousands of years ago, looking at the people below. They’re barely scraping out a living in a hostile territory, doing some hunting, some fishing, and a little planting. Off in the distance, you see the glitter of a glacier. Moving closer, you see that it’s melting fast and that it’s barely damming a huge lake. As you watch, the lake breaks through, sweeping down a riverbed, carving it deeper, splashing up against cliffs on the far side of the landscape—some of which collapse.
当您观看时,茫然的居民开始探索开口。在另一边,有一片郁郁葱葱的风景,与他们从未见过的体型更大的动物成群结队,有些在长着巨大种子头的草地上吃草,有些则在一堆堆掉落的水果上争吵。
As you watch, the dazed inhabitants begin to explore the opening. On the other side, there’s a lush landscape, teaming with bigger animals than they’ve ever seen before, some grazing on grass with huge seed heads, some squabbling over mounds of fallen fruit.
人们搬进来了。几乎立刻,他们开始过上更好的生活。但是随着岁月的流逝,您会看到他们适应了。他们开始用网在湍急的溪流中捕鱼。他们学会了打倒大型动物所需的团队合作,尽管一路上也有一些人死亡。他们找到了更好的方法来种植这种他们称之为“小麦”的新草。
People move in. Almost immediately, they begin to live better. But as the years fly past, you see them adapt. They begin to use nets to fish in the fast-running streams. They learn the teamwork needed to bring down the larger animals, though not without a few deaths along the way. They find ever-better ways to cultivate this new grass they’ve come to call “wheat.”
正如你所看到的,疯狂的创新让位于稳定的解决方案,一种在这片新土地上生活的好方法,一种传授给每一代新人的方法。虽然就在那边,你发现有人在发明轮子......
As you watch, the mad burst of innovation gives way to a stable solution, a good way to live in this new land, a way that’s taught to each new generation. Although just over there, you spy someone inventing the wheel ...
在本世纪初,敏捷方法的采用有时看起来像是一个巨大的堤坝决堤,开辟了一条通向更好——更高效、更快乐——的软件开发方式的途径。许多早期采用者立即看到了好处,即使他们几乎不知道自己在做什么。
In the early years of this century, the adoption of Agile methods sometimes seemed like a vast dam breaking, opening up a way to a better—more productive, more joyful—way of developing software. Many early adopters saw benefits right away, even though they barely knew what they were doing.
有些人比其他人更容易。程序员就像上面寓言中的猎人。是的,他们必须学习新技能才能猎杀野牛,但他们或多或少知道如何猎杀兔子,而且附近有很多兔子。测试人员更像是在用鱼叉捕鱼不起作用的土地上的鱼叉捕鱼者。从用鱼叉捕鱼到用网捕鱼是一个比从兔子到野牛更大的概念跳跃。而且,虽然某些技能(例如清理鱼)在新土地上是相同的,但测试人员必须发明新的织网技能才能真正发挥作用。
Some had an easier time of it than others. Programmers were like the hunters in the fable above. Yes, they had to learn new skills in order to hunt bison, but they knew how to hunt rabbits, more or less, and there were plenty of rabbits around. Testers were more like spear-fishers in a land where spear-fishing wouldn’t work. Going from spear-fishing to net-fishing is a much bigger conceptual jump than going from rabbit to bison. And, while some of the skills—cleaning fish, for example—were the same in the new land, the testers had to invent new skills of net-weaving before they could truly pull their weight.
所以测试滞后了。幸运的是,我们有像 Lisa 和 Janet 这样的早期采用者,他们与程序员、不嫉妒自己的角色或独立性的测试人员一起投入其中,他们是非常令人愉快的人,他们能够找出敏捷测试中最大的变化:测试人员的新的社会角色。
So testing lagged behind. Fortunately, we had early adopters like Lisa and Janet, people who dove right in alongside the programmers, testers who were not jealous of their role or their independence, downright pleasant people who could figure out the biggest change of all in Agile testing: the tester’s new social role.
结果,我们有了这本书。这是稳定的解决方案,是测试人员在我们这个新的敏捷领域生活的好方法。这不是最后的决定——我们可以使用轮子,我自己也渴望有人发明抗生素——但这里所教的内容将对你有用,直到有人,也许是丽莎和珍妮特,带来下一个巨大的变化。
As a result, we have this book. It’s the stable solution, the good way for testers to live in this new Agile land of ours. It’s not the final word—we could use the wheel, and I myself am eager for someone to invent antibiotics—but what’s taught here will serve you well until someone, perhaps Lisa and Janet, brings the next big change.
我们是极限编程 (XP) 的早期采用者,在 XP 团队中进行测试,这些团队根本不确定测试人员或他们的测试品牌适合在哪里。当时,敏捷(不称为agile yet)有关验收测试或专业测试人员如何做出贡献的文献。我们不仅从自己的经验中学习,而且从小型敏捷社区中的其他人那里学习。2002 年,在Janet 的大力帮助下,Lisa 与 Tip House 合着了《测试极限编程》 。从那时起,敏捷开发得到发展,敏捷测试社区蓬勃发展。有这么多人贡献想法,我们学到了很多关于敏捷测试的知识。
We were early adopters of Extreme Programming (XP), testing on XP teams that weren’t at all sure where testers or their brand of testing fit in. At the time, there wasn’t much in the agile (which wasn’t called agile yet) literature about acceptance testing, or how professional testers might contribute. We learned not only from our own experiences but from others in the small agile community. In 2002, Lisa co-wrote Testing Extreme Programming with Tip House, with lots of help from Janet. Since then, agile development has evolved, and the agile testing community has flourished. With so many people contributing ideas, we’ve learned a whole lot more about agile testing.
我们单独和一起帮助团队过渡到敏捷,帮助测试人员学习如何为敏捷团队做出贡献,并与敏捷社区中的其他人一起探索敏捷团队在测试中更成功的方法。我们的经历不同。Lisa 大部分时间都在稳定的团队中担任敏捷测试员,多年来一直在零售、电话和金融行业的 Web 应用程序上工作。Janet 曾与多个行业的软件组织合作开发企业系统。这些敏捷项目包括开发消息处理系统、环境跟踪系统、远程数据管理系统(包括嵌入式应用程序、通信网络和应用程序)、油气生产核算应用程序、以及在航空运输业中的应用。她扮演过不同的角色——有时是测试员,有时是教练——但她一直致力于更好地将测试员与团队的其他成员结合起来。她在团队中工作的时间短则六个月,长则一年半。
Individually and together, we’ve helped teams transition to agile, helped testers learn how to contribute on agile teams, and worked with others in the agile community to explore ways that agile teams can be more successful at testing. Our experiences differ. Lisa has spent most of her time as an agile tester on stable teams working for years at a time on web applications in the retail, telephony, and financial industries. Janet has worked with software organizations developing enterprise systems in a variety of industries. These agile projects have included developing a message-handling system, an environmental-tracking system, a remote data management system (including an embedded application, with a communication network as well as the application), an oil and gas production accounting application, and applications in the airline transportation industry. She has played different roles—sometimes tester, sometimes coach—but has always worked to better integrate the testers with the rest of the team. She has been with teams from as little as six months to as long as one-and-a-half years.
有了这些不同的观点,我们学会了一起工作并互补彼此的技能,并且我们一起提供了许多演示和教程。
With these different points of view, we have learned to work together and complement each other’s skill sets, and we have given many presentations and tutorials together.
已经出版了几本关于测试和测试模式的敏捷开发的优秀书籍(请参阅我们的参考书目)。这些书通常侧重于帮助开发人员。我们决定写一本书,旨在帮助敏捷团队使用业务可以理解的测试更成功地交付业务价值。我们希望帮助使用更传统的开发方法的测试人员和质量保证 (QA) 专业人员过渡到敏捷开发。
Several excellent books oriented toward agile development on testing and test patterns have been published (see our Bibliography). These books are generally focused on helping the developer. We decided to write a book aimed at helping agile teams be more successful at delivering business value using tests that the business can understand. We want to help testers and quality assurance (QA) professionals who have worked in more traditional development methodologies make the transition to agile development.
我们已经弄清楚如何在实际的、日常的层面上应用我们自己与各种规模的团队合作的经验成果以及其他敏捷实践者的各种想法。我们将所有这些内容汇总在本书中,以帮助测试人员、质量保证经理、开发人员、开发经理、产品所有者以及任何与敏捷项目的有效测试相关的人交付他们的客户需要的软件。然而,我们关注的是测试人员的角色,一个可以被各种专业人士采用的角色。
We’ve figured out how to apply—on a practical, day-to-day level—the fruits of our own experience working with teams of all sizes and a variety of ideas from other agile practitioners. We’ve put all this together in this book to help testers, quality assurance managers, developers, development managers, product owners, and anyone else with a stake in effective testing on agile projects to deliver the software their customers need. However, we’ve focused on the role of the tester, a role that may be adopted by a variety of professionals.
敏捷测试实践并不局限于敏捷团队的成员。它们也可用于改进使用传统开发方法的项目的测试。本书还旨在帮助测试人员使用任何类型的开发方法从事项目。
Agile testing practices aren’t limited to members of agile teams. They can be used to improve testing on projects using traditional development methodologies as well. This book is also intended to help testers working on projects using any type of development methodology.
敏捷开发并不是成功交付软件的唯一途径。然而,我们所处的所有成功团队,无论是敏捷团队还是瀑布团队,都有几个关键的共性。程序员编写并自动化单元和集成测试,以提供良好的代码覆盖率。他们在使用源代码控制和代码集成方面受到纪律处分。熟练的测试人员从开发周期的开始就参与进来,并有时间和资源来完成所有必要形式的测试。定期运行和检查涵盖更高级别系统功能的自动回归套件。开发团队了解客户的工作和需求,并与业务专家密切合作。
Agile development isn’t the only way to successfully deliver software. However, all of the successful teams we’ve been on, agile or waterfall, have had several critical commonalities. The programmers write and automate unit and integration tests that provide good code coverage. They are disciplined in the use of source code control and code integration. Skilled testers are involved from the start of the development cycle and are given time and resources to do an adequate job of all necessary forms of testing. An automated regression suite that covers the system functionality at a higher level is run and checked regularly. The development team understands the customers’ jobs and their needs, and works closely together with the business experts.
使项目成功的是人,而不是方法论或工具。我们喜欢敏捷开发,因为它的价值观、原则和核心实践使人们能够做到最好,而测试和质量是敏捷开发的核心。在本书中,我们将解释如何将敏捷价值观和原则应用于您独特的测试情况,并使您的团队取得成功。我们在第 1 章“什么是敏捷测试?”中有更多相关内容。”和第 2 章“敏捷测试人员的十项原则”。
People, not methodologies or tools, make projects successful. We enjoy agile development because its values, principles, and core practices enable people to do their best work, and testing and quality are central to agile development. In this book, we explain how to apply agile values and principles to your unique testing situation and enable your teams to succeed. We have more about that in Chapter 1, “What Is Agile Testing, Anyway?” and in Chapter 2, “Ten Principles for Agile Testers.”
在体验了敏捷开发的好处之后,我们使用敏捷实践来制作这本书。当我们开始写这本书时,我们与来自世界各地的敏捷测试人员和团队进行了交谈,以了解他们遇到了什么问题以及他们是如何解决这些问题的。我们计划了如何在本书中涵盖这些领域。
Having experienced the benefits of agile development, we used agile practices to produce this book. As we began work on the book, we talked to agile testers and teams from around the globe to find out what problems they encountered and how they addressed them. We planned how we would cover these areas in the book.
我们制定了基于两周迭代的发布计划。每两周,我们就会向我们的图书网站发送两章草稿。因为我们不在同一地点,所以我们找到了用于交流的工具,为我们的章节提供“源代码控制”,将产品交付给我们的客户,并获得他们的反馈。我们不能实时“配对”太多,但我们来回交换章节以进行复习和修订,并且每天通过即时消息进行非正式的“站会”。
We made a release plan based on two-week iterations. Every two weeks, we delivered two rough-draft chapters to our book website. Because we aren’t co-located, we found tools to use to communicate, provide “source code control” for our chapters, deliver the product to our customers, and get their feedback. We couldn’t “pair” much real-time, but we traded chapters back and forth for review and revision, and had informal “stand-ups” daily via instant message.
我们的“客户”是敏捷社区中的慷慨人士,他们自愿审阅章节草稿。他们通过电子邮件或(如果我们幸运的话)亲自提供反馈。在我们继续写作和修改时,我们使用反馈来指导我们。在完成所有草稿后,我们制定了一个新的计划来完成修订,并结合了我们“客户”的所有有用想法。
Our “customers” were the generous people in the agile community who volunteered to review draft chapters. They provided feedback by email or (if we were lucky) in person. We used the feedback to guide us as we continued writing and revising. After all the rough drafts were done, we made a new plan to complete the revisions, incorporating all the helpful ideas from our “customers.”
我们最重要的工具是思维导图。我们首先创建了一张思维导图,展示了我们如何设想整本书。然后我们为本书的每个部分创建了思维导图。在写每一章之前,我们用思维导图进行头脑风暴。在我们修订的过程中,我们重新审视了思维导图,这有助于我们思考我们可能遗漏的想法。
Our most important tool was mind maps. We started out by creating a mind map of how we envisioned the whole book. We then created mind maps for each section of the book. Before writing each chapter, we brainstormed with a mind map. As we revised, we revisited the mind maps, which helped us think of ideas we may have missed.
因为我们认为思维导图增加了很多价值,所以我们将思维导图作为每章开头的一部分。我们希望它们能帮助您大致了解本章中包含的所有信息,并启发您自己尝试使用思维导图。
Because we think the mind maps added so much value, we’ve included the mind map as part of the opening of each chapter. We hope they’ll help you get an overview of all the information included in the chapter, and inspire you to try using mind maps yourself.
如果您曾经问过以下任何我们已经多次听到的极好的问题,本书将对您有所帮助:
This book will help you if you’ve ever asked any of the following excellent questions, which we’ve heard many times:
如果开发人员正在编写测试,那么测试人员会做什么?
If developers are writing tests, what do the testers do?
我是一名 QA 经理,我们公司正在实施敏捷开发(Scrum、XP、DSDM,随你喜欢)。我现在的角色是什么?
I’m a QA manager, and our company is implementing agile development (Scrum, XP, DSDM, name your flavor). What’s my role now?
我曾在一个传统的瀑布团队中担任测试员,我对所读到的关于敏捷的内容感到非常兴奋。在敏捷团队中工作我需要知道什么?
I’ve worked as a tester on a traditional waterfall team, and I’m really excited by what I’ve read about agile. What do I need to know to work on an agile team?
什么是“敏捷测试员”?
What’s an “agile tester”?
我是一个敏捷团队的开发人员。我们正在编写测试优先的代码,但我们的客户仍然对我们提供的产品不满意。我们缺少什么?
I’m a developer on an agile team. We’re writing code test-first, but our customers still aren’t happy with what we deliver. What are we missing?
我是一个敏捷团队的开发人员。我们正在编写测试优先的代码。我们确保对所有代码进行测试。为什么我们需要测试人员?
I’m a developer on an agile team. We’re writing our code test-first. We make sure we have tests for all our code. Why do we need testers?
我指导一个敏捷开发团队。我们的 QA 团队跟不上我们,测试总是落后。我们应该只计划测试开发背后的迭代吗?
I coach an agile development team. Our QA team can’t keep up with us, and testing always lags behind. Should we just plan to test an iteration behind development?
我是一名软件开发经理。我们最近过渡到敏捷,但我们所有的测试人员都退出了。为什么?
I’m a software development manager. We recently transitioned to agile, but all our testers quit. Why?
我是一个敏捷团队的测试员。我没有任何编程或自动化技能。在敏捷团队中有适合我的位置吗?
I’m a tester on a team that’s going agile. I don’t have any programming or automation skills. Is there any place for me on an agile team?
测试如何跟上两周的迭代?
How can testing possibly keep up with two-week iterations?
负载测试、性能测试、可用性测试以及所有其他“能力”呢?这些适合放在哪里?
What about load testing, performance testing, usability testing, all the other “ilities”? Where do these fit in?
我们有审计要求。敏捷开发和测试如何解决这些问题?
We have audit requirements. How does agile development and testing address these?
如果您有类似的问题,并且正在寻找有关测试人员如何为敏捷团队做出贡献以及敏捷团队如何有效进行测试的实用建议,那么您选对了这本书。
If you have similar questions and you’re looking for practical advice about how testers contribute to agile teams and how agile teams can do an effective job of testing, you’ve picked up the right book.
敏捷开发有很多“风格”,但它们都有很多共同点。我们支持敏捷宣言,我们在第 1 章“什么是敏捷测试?”中对此进行了解释。” 无论您是在练习 Scrum、极限编程、Crystal、DSDM,还是您自己的敏捷开发变体,您都可以在这里找到有助于测试工作的信息。
There are many “flavors” of agile development, but they all have much in common. We support the Agile Manifesto, which we explain in Chapter 1, “What Is Agile Testing, Anyway?” Whether you’re practicing Scrum, Extreme Programming, Crystal, DSDM, or your own variation of agile development, you’ll find information here to help with your testing efforts.
如果您不确定从何处开始阅读本书,或者您只想快速了解一下,我们建议您阅读最后一章,即第 21 章“成功的关键因素”,并按照它引导您的方向进行操作。
If you aren’t sure where to start in this book, or you just want a quick overview, we suggest you read the last chapter, Chapter 21, “Key Success Factors,” and follow wherever it leads you.
如果您想快速回答诸如“敏捷测试与瀑布项目测试有何不同?”等问题 或者“传统团队的测试人员和敏捷测试人员有什么区别?”,从第 I 部分开始,其中包括以下章节:
If you want quick answers to questions such as “Is agile testing different than testing on waterfall projects?” or “What’s the difference between a tester on a traditional team and an agile tester?,” start with Part I, which includes the following chapters:
这些章节是 Robin 在他的用户故事中要求的“冰山一角”。它们概述了敏捷与传统的分阶段方法有何不同,并探索了“整个团队”的质量和测试方法。
These chapters are the “tip of the iceberg” that Robin requested in his user story. They include an overview of how agile differs from a traditional phased approach and explore the “whole team” approach to quality and testing.
在本书的这一部分,我们定义了“敏捷测试思维模式”以及是什么让测试人员在敏捷团队中取得成功。我们解释了测试人员如何应用敏捷价值观和原则来贡献他们的特殊专业知识。
In this part of the book we define the “agile testing mind-set” and what makes testers successful on agile teams. We explain how testers apply agile values and principles to contribute their particular expertise.
如果您是传统 QA 团队的测试员或经理,或者您正在指导一个正在转向敏捷的团队,那么第II 部分将帮助您应对转型中的团队所面临的组织挑战。“整个团队”的态度代表了团队成员的许多文化变化,但它有助于克服测试人员在怀疑他们将拥有多少控制权或是否需要编写代码时产生的恐惧。
If you’re a tester or manager on a traditional QA team, or you’re coaching a team that’s moving to agile, Part II will help you with the organizational challenges faced by teams in transition. The “whole team” attitude represents a lot of cultural changes to team members, but it helps overcome the fear testers have when they wonder how much control they’ll have or whether they’ll be expected to write code.
第二部分回答的一些问题是:
Some of the questions answered in Part II are:
我们如何让质量检查团队参与进来?
How can we engage the QA team?
管理层的期望如何?
What about management’s expectations?
我们应该如何构建我们的敏捷团队,测试人员适合在哪里?
How should we structure our agile team, and where do the testers fit?
招聘敏捷测试人员时我们看重什么?
What do we look for when hiring an agile tester?
我们如何应对分布在全球各地的团队?
How do we cope with a team distributed across the globe?
第二部分还介绍了一些我们并不总是喜欢谈论的话题。我们探索有关如何转换传统环境中常见的流程和模型的想法,例如审计或 SOX 合规性。
Part II also introduces some topics we don’t always enjoy talking about. We explore ideas about how to transition processes and models, such as audits or SOX compliance, that are common in traditional environments.
指标及其应用方式可能是一个有争议的问题,但有一些积极的方法可以使用它们来使团队受益。缺陷跟踪很容易成为团队争论的焦点,诸如“我们是否使用缺陷跟踪系统?”之类的问题。或者“我们什么时候记录错误?”
Metrics and how they’re applied can be a controversial issue, but there are positive ways to use them to benefit the team. Defect tracking easily becomes a point of contention for teams, with questions such as “Do we use a defect-tracking system?” or “When do we log bugs?”
具有传统测试团队经验的人员关于敏捷测试的两个常见问题是“测试计划如何?” 和“真的没有关于敏捷项目的文档吗?” 第二部分揭开这些谜团。
Two common questions about agile testing from people with traditional test team experience are “What about test plans?” and “Is it true there’s no documentation on agile projects?” Part II clears up these mysteries.
第二部分各章如下:
The chapters in Part II are as follows:
您是否需要有关在敏捷项目上进行哪些类型的测试的更多详细信息?你想知道谁做了什么测试吗?你怎么知道你是否已经完成了所有需要的测试?您如何决定哪些实践、技术和工具适合您的特定情况?如果这些是您的顾虑,请查看第 III 部分。
Do you want more details on what types of testing are done on agile projects? Are you wondering who does what testing? How do you know whether you’ve done all the testing that’s needed? How do you decide what practices, techniques, and tools fit your particular situation? If these are your concerns, check out Part III.
我们使用 Brian Marick 的敏捷测试象限来解释测试的目的。象限可帮助您定义测试应解决的所有不同领域,从单元级测试到可靠性和其他“能力”,以及介于两者之间的一切。这就是我们深入了解如何交付高质量产品的关键所在。我们解释了可以帮助您与客户进行良好沟通并更好地了解他们的需求的技术。本书的这一部分展示了测试如何在多个层面推动开发。它还为您的工具包提供了工具,可以帮助您有效地定义、设计和执行支持团队和评价产品的测试。这些章节包括以下内容:
We use Brian Marick’s Agile Testing Quadrants to explain the purpose of testing. The quadrants help you define all the different areas your testing should address, from unit level tests to reliability and other “ilities,” and everything in between. This is where we get down into the nitty-gritty of how to deliver a high-quality product. We explain techniques that can help you to communicate well with your customers and better understand their requirements. This part of the book shows how tests drive development at multiple levels. It also provides tools for your toolkit that can help you to effectively define, design, and execute tests that support the team and critique the product. The chapters include the following:
Chapter 6: The Purpose of Testing
Chapter 7: Technology-Facing Tests that Support the Team
Chapter 8: Business-Facing Tests that Support the Team
Chapter 9: Toolkit for Business-Facing Tests that Support the Team
Chapter 10: Business-Facing Tests that Critique the Product
Chapter 11: Critiquing the Product Using Technology-Facing Tests
测试自动化是成功的敏捷团队的核心焦点,它对很多人来说都是一个可怕的话题(我们知道,因为它让我们之前感到害怕!)。您如何将测试自动化压缩到短迭代中并仍然完成所有故事?
Test automation is a central focus of successful agile teams, and it’s a scary topic for lots of people (we know, because it’s had us running scared before!). How do you squeeze test automation into short iterations and still get all the stories completed?
第 IV 部分详细介绍了何时以及为何进行自动化、如何克服测试自动化的障碍以及如何开发和实施适合您的团队的测试自动化策略。由于测试自动化工具变化和发展如此迅速,我们的目的不是解释如何使用特定工具,而是帮助您选择和使用适合您情况的正确工具。我们的敏捷测试自动化技巧将帮助您应对困难的挑战,例如测试遗留代码。
Part IV gets into the details of when and why to automate, how to overcome barriers to test automation, and how to develop and implement a test automation strategy that works for your team. Because test automation tools change and evolve so rapidly, our aim is not to explain how to use specific tools, but to help you select and use the right tools for your situation. Our agile test automation tips will help you with difficult challenges such as testing legacy code.
章节如下:
The chapters are as follows:
如果您只是想了解测试人员在整个敏捷开发周期中所做的事情,或者您需要帮助整理本书中的所有信息,请转到第 V 部分。在这里,我们记录了敏捷测试人员生活中的一次迭代,以及更多。测试人员在整个敏捷软件开发周期中贡献了巨大的价值。在第五部分,我们解释了测试人员每天进行的活动。我们从规划发布和迭代开始,让每个迭代都有一个良好的开端,然后通过迭代——与客户和开发团队协作、测试和编写代码。我们通过交付新功能并为团队寻找改进流程的方法来结束迭代。
If you just want to get a feel for what testers do throughout the agile development cycle, or you need help putting together all the information in this book, go to Part V. Here we chronicle an iteration, and more, in the life of an agile tester. Testers contribute enormous value throughout the agile software development cycles. In Part V, we explain the activities that testers do on a daily basis. We start with planning releases and iterations to get each iteration off to a good start, and move through the iteration—collaborating with the customer and development teams, testing, and writing code. We end the iteration by delivering new features and finding ways for the team to improve the process.
这些章节是这样分解的:
The chapters break down this way:
在第 21 章“关键成功因素”中,我们介绍了敏捷团队可用于成功测试的七个关键因素。如果您在决定从哪里开始敏捷测试或如何改进您现在所做的事情时遇到困难,这些成功因素将为您指明方向。
In Chapter 21, “Key Success Factors,” we present seven key factors agile teams can use for successful testing. If you’re having trouble deciding where to start with agile testing, or how to work on improving what you’re doing now, these success factors will give you some direction.
我们还提供了一个词汇表,希望您能从中找到有用的词汇表,以及参考书目中的书籍、文章、网站和博客。
We’ve also included a glossary we hope you will find useful, as well as references to books, articles, websites, and blogs in the bibliography.
敏捷开发就是把工作做到最好。每个团队都有独特的挑战。我们已尝试提供所有我们认为可能对敏捷测试人员、他们的团队、经理和客户有帮助的信息。应用您认为适合您情况的技巧。不断试验,评估结果,然后回过头来看这本书,看看有什么可以帮助你改进的。我们的目标是帮助测试人员和敏捷团队尽其所能交付最好、最有价值的产品。
Agile development is all about doing your best work. Every team has unique challenges. We’ve tried to present all the information that we think may help agile testers, their teams, managers, and customers. Apply the techniques that you think are appropriate for your situation. Experiment constantly, evaluate the results, and come back to this book to see what might help you improve. Our goal is to help testers and agile teams enjoy delivering the best and most valuable product they can.
当我们询问 Canoo WebTest 的创始人兼项目经理 Dierk König 时,他认为敏捷测试的首要成功因素是什么,他回答说:“从今天开始!” 您现在可以迈出一小步来改进团队的测试。开始吧!
When we asked Dierk König, founder and project manager of Canoo WebTest, what he thought was the number one success factor for agile testing, he answered: “Start doing it—today!” You can take a baby step to improve your team’s testing right now. Go get started!
有太多人帮助我们完成了这本书,以至于很难知道首先要感谢谁。Chris Guzikowski 给了我们写这本书的机会,并一直鼓励我们。当我们决定是否承担如此艰巨的任务时,迈克·科恩 (Mike Cohn) 给了我们一个明智的建议,即写书的最佳理由是您有话要说。关于敏捷测试,我们肯定有很多话要说。幸运的是,还有很多其他愿意帮助我们的人。
So many people have helped us with this book that it’s hard to know whom to thank first. Chris Guzikowski gave us the opportunity to write this book and kept encouraging us along the way. When we were deciding whether to take on such a mammoth task, Mike Cohn gave us the sage advice that the best reason to write a book is that you have something to say. We sure have lots to say about agile testing. Fortunately, so do lots of other people who were willing to lend us a hand.
非常感谢 Brian Marick 和 Mike Cohn 撰写了如此友善的前言。我们很荣幸 Mike 选择我们的书作为他的签名系列。我们感谢他在本书中包含的许多想法和观察。
Many thanks to Brian Marick and Mike Cohn for writing such kind forewords. We’re honored that Mike selected our book for his signature series. We’re grateful for the many ideas and observations of his that are included in this book.
Brian Marick 的“敏捷测试矩阵”多年来一直在指导我们的敏捷项目,它提供了第 III 部分的核心内容。感谢您,Brian,感谢您提出象限(以及对敏捷测试的许多其他贡献)并让我们在这里使用它们。
Brian Marick’s “Agile Testing Matrix” has guided both of us in our agile projects for several years, and it provides the core of Part III. Thank you, Brian, for thinking up the quadrants (and so many other contributions to agile testing) and letting us use them here.
我们不断利用反馈的敏捷价值。非常感谢我们的官方审稿人:Jennitta Andrea、Gerard Meszaros、Ron Jeffries 和 Paul Duvall。每个人都有独特而有见地的评论,帮助我们大大改进了本书。Gerard 还帮助我们在测试术语方面更加一致和正确,并贡献了一些敏捷测试的成功案例。
We made constant use of the agile value of feedback. Many thanks to our official reviewers: Jennitta Andrea, Gerard Meszaros, Ron Jeffries, and Paul Duvall. Each one had unique and insightful comments that helped us greatly improve the book. Gerard also helped us be more consistent and correct in our testing terminology, and contributed some agile testing success stories.
特别感谢两位审阅者和一流的敏捷测试人员,他们阅读了我们写的每一个字,并花费数小时亲自与我们讨论了章节草稿:Pierre Veragen 和 Paul Rogers。本书中的许多好点子都是他们的。
Special thanks to two reviewers and top-notch agile testers who read every word we wrote and spent hours discussing the draft chapters with us in person: Pierre Veragen and Paul Rogers. Many of the good ideas in this book are theirs.
我们采访了几个团队,了解他们会给新的敏捷团队和测试人员什么建议,并从敏捷测试社区的同事那里征求成功案例和“经验教训”。衷心感谢我们的侧边栏和引用的众多贡献者,以及有用反馈的提供者,包括(排名不分先后)Robin Dymond、Bret Pettichord、Tae Chang、Bob Galen、Erika Boyer、Grig Gheorghiu、Erik Bos、Mark Benander,乔纳森·拉斯穆森、安迪·波尔斯、迪尔克·科尼格、拉斐尔·桑托斯、杰森·霍尔泽、克里斯托夫·卢维翁、大卫·里德、约翰·沃里斯、克里斯·麦克马洪、德克兰·惠兰、迈克尔·博尔顿、伊丽莎白·亨德里克森、乔·亚基奇、安德鲁·格洛弗、亚历山德罗·科里诺、科尼·塔塔利亚、马库斯·加特纳, 梅根·萨姆雷尔, 内森·西尔伯曼, 迈克·托马斯, 迈克·布斯, 史蒂夫·珀金斯, 约瑟夫·金, 雅库布·奥莱兹凯维奇, 皮埃尔·韦拉根 (再次), 保罗·罗杰斯 (再次),
We interviewed several teams to learn what advice they would give new agile teams and testers, and solicited success stories and “lessons learned” from colleagues in the agile testing community. Heartfelt thanks to our many contributors of sidebars and quotes, as well as providers of helpful feedback, including (in no particular order) Robin Dymond, Bret Pettichord, Tae Chang, Bob Galen, Erika Boyer, Grig Gheorghiu, Erik Bos, Mark Benander, Jonathan Rasmusson, Andy Pols, Dierk König, Rafael Santos, Jason Holzer, Christophe Louvion, David Reed, John Voris, Chris McMahon, Declan Whelan, Michael Bolton, Elisabeth Hendrickson, Joe Yakich, Andrew Glover, Alessandro Collino, Coni Tartaglia, Markus Gärtner, Megan Sumrell, Nathan Silberman, Mike Thomas, Mike Busse, Steve Perkins, Joseph King, Jakub Oleszkiewicz, Pierre Veragen (again), Paul Rogers (again), Jon Hagar, Antony Marcano, Patrick Wilson-Welsh, Patrick Fleisch, Apurva Chandra, Ken De Souza, and Carol Vaage.
也非常感谢我们社区的其他非官方审稿人,他们阅读了章节,提供了反馈和想法,并让我们从他们身上汲取灵感,包括 Tom Poppendieck、Jun Bueno、Kevin Lawrence、Hannu Kokko、Titus Brown、Wim van de Goor , Lucas Campos, Kay Johansen, Adrian Howard, Henrik Kniberg, Shelly Park, Robert Small, Senaka Suriyaachchi, and Erik Petersen. 如果我们忘记在这里列出您,那并不是因为我们对您的贡献的重视程度有所降低,只是因为我们没有保留足够好的笔记!我们希望您会在完成的书中看到您的时间和努力是如何得到回报的。
Many thanks also to the rest of our community of unofficial reviewers who read chapters, gave feedback and ideas, and let us bounce ideas off of them, including Tom Poppendieck, Jun Bueno, Kevin Lawrence, Hannu Kokko, Titus Brown, Wim van de Goor, Lucas Campos, Kay Johansen, Adrian Howard, Henrik Kniberg, Shelly Park, Robert Small, Senaka Suriyaachchi, and Erik Petersen. And if we’ve neglected to list you here, it’s not that we value your contribution any less, it’s just that we didn’t keep good enough notes! We hope you will see how your time and effort paid off in the finished book.
我们感谢帮助我们和我们的团队通过敏捷取得成功的敏捷先驱们奠定的基础。您会在参考书目中找到他们的一些作品。我们感谢敏捷团队,他们为我们提供了如此多的开源测试工具,帮助我们所有的团队交付如此多的价值。其中一些工具也列在参考书目中。
We appreciate the groundwork laid by the agile pioneers who have helped us and our teams succeed with agile. You’ll find some of their works in the bibliography. We are grateful for the agile teams that have given us so many open source test tools that help all of our teams deliver so much value. Some of those tools are also listed in the bibliography.
感谢 Mike Thomas 拍摄了本书中出现的许多敏捷团队的动作照片。我们希望这些照片向那些刚接触敏捷测试和开发的人表明,没有什么大奥秘——只是好人聚在一起讨论、演示和画图。
Thanks to Mike Thomas for taking many of the action photos of an agile team that appear in this book. We hope these photos show those of you new to agile testing and development that there’s no big mystery—it’s just good people getting together to discuss, demo, and draw pictures.
非常感谢我们的 Addison-Wesley 编辑和制作团队,他们耐心地回答了许多问题,并将这本书变成了您在这里看到的具有专业外观的书,其中包括 Raina Chrobak、Chris Zahn、John Fuller、Sally Gregg、Bonnie Granat、Diane Freed、Jack刘易斯和金阿尼。
Thanks so much to our Addison-Wesley editorial and production team who patiently answered many questions and turned this into the professional-looking book you see here, including Raina Chrobak, Chris Zahn, John Fuller, Sally Gregg, Bonnie Granat, Diane Freed, Jack Lewis, and Kim Arney.
Lisa Crispin是一名敏捷测试实践者和教练。她擅长向测试人员和敏捷团队展示测试人员如何增加价值以及如何通过面向业务的测试来指导开发。她的使命是为软件测试世界带来敏捷的乐趣,为敏捷开发世界带来测试的乐趣。Lisa 于 2000 年加入了她的第一个敏捷团队,多年来担任过程序员、分析师、测试员和 QA 主管。自 2003 年以来,她一直是 ePlan Services, Inc. 的 Scrum/XP 团队的测试员。她经常在北美和欧洲的会议上主持有关敏捷测试的教程和研讨会。Lisa 定期为Better Software杂志、 IEEE Software和Methods and Tools等出版物撰写有关敏捷测试的文章. Lisa 还与 Tip House 合着了《测试极限编程》(Addison-Wesley,2002 年)。
Lisa Crispin is an agile testing practitioner and coach. She specializes in showing testers and agile teams how testers can add value and how to guide development with business-facing tests. Her mission is to bring agile joy to the software testing world and testing joy to the agile development world. Lisa joined her first agile team in 2000, having enjoyed many years working as a programmer, analyst, tester, and QA director. Since 2003, she’s been a tester on a Scrum/XP team at ePlan Services, Inc. She frequently leads tutorials and workshops on agile testing at conferences in North America and Europe. Lisa regularly contributes articles about agile testing to publications such as Better Software magazine, IEEE Software, and Methods and Tools. Lisa also co-authored Testing Extreme Programming (Addison-Wesley, 2002) with Tip House.
有关 Lisa 工作的更多信息,请访问她的网站www.lisacrispin.com和www.agiletester.ca,或发送电子邮件至lisa@agiletester.ca。
For more about Lisa’s work, visit her websites, www.lisacrispin.com and www.agiletester.ca, or email her at lisa@agiletester.ca.
Janet Gregory是敏捷质量流程咨询和培训公司 DragonFire Inc. 的创始人。她热衷于帮助团队建立质量体系。在过去的十年里,她担任过教练和测试员,将敏捷实践引入了大公司和小公司。她的重点是与业务用户和测试人员合作,以了解他们在敏捷项目中的角色。当 Janet 与她的敏捷团队中的开发人员合作实施创新的敏捷测试自动化解决方案时,她的编程背景无疑是一个优势。Janet 经常在敏捷和测试软件会议上发表演讲,她是北美敏捷测试社区的主要贡献者。
Janet Gregory is the founder of DragonFire Inc., an agile quality process consultancy and training firm. Her passion is helping teams build quality systems. For the past ten years, she has worked as a coach and tester, introducing agile practices into both large and small companies. Her focus is working with business users and testers to understand their role in agile projects. Janet’s programming background is a definite plus when she partners with developers on her agile teams to implement innovative agile test automation solutions. Janet is a frequent speaker at agile and testing software conferences, and she is a major contributor to the North American agile testing community.
有关 Janet 工作的更多信息,请访问她的网站www.janetgregory.ca、www.janetgregory.blogspot.com和www.agiletester.ca,或者您可以发送电子邮件至janet@agiletester.ca。
For more about Janet’s work, visit her websites at www.janetgregory.ca, www.janetgregory.blogspot.com, and www.agiletester.ca, or you can email her at janet@agiletester.ca.
在前两章中,我们概述了敏捷测试,强调了敏捷测试与传统的分阶段或“瀑布”方法中的测试有何不同。我们探索“整个团队”的质量和测试方法。
In the first two chapters, we provide an overview of agile testing, highlighting how agile testing differs from testing in a traditional phased or “waterfall” approach. We explore the “whole team” approach to quality and testing.
就像许多术语一样,“敏捷开发”和“敏捷测试”对不同的人有不同的含义。在本章中,我们解释了我们对敏捷的看法,它反映了敏捷宣言以及不同敏捷方法所共有的一般原则和价值观。我们想与您(读者)分享共同语言,因此我们将复习一些词汇。我们将敏捷开发和测试与更传统的分阶段方法进行比较和对比。敏捷开发提倡的“整个团队”方法是我们对质量和测试的态度的核心,所以我们在这里也谈到了这一点。
Like a lot of terminology, “agile development” and “agile testing” mean different things to different people. In this chapter, we explain our view of agile, which reflects the Agile Manifesto and general principles and values shared by different agile methods. We want to share a common language with you, the reader, so we’ll go over some of our vocabulary. We compare and contrast agile development and testing with the more traditional phased approach. The “whole team” approach promoted by agile development is central to our attitude toward quality and testing, so we also talk about that here.
“敏捷”是一个流行词,可能有一天会不再使用,并使这本书看起来过时了。它具有适用于不同情况的不同含义。定义“敏捷开发”的一种方法是查看敏捷宣言(见图1-1)。
“Agile” is a buzzword that will probably fall out of use someday and make this book seem obsolete. It’s loaded with different meanings that apply in different circumstances. One way to define “agile development” is to look at the Agile Manifesto (see Figure 1-1).
使用宣言中的价值观来指导我们,我们努力在极短的发布周期内提供小块的商业价值。
Using the values from the Manifesto to guide us, we strive to deliver small chunks of business value in extremely short release cycles.
我们在本书中从广义上使用“敏捷”一词。无论您的团队是在实践一种特定的敏捷方法,例如 Scrum、XP、Crystal、DSDM 或 FDD,仅举几例,还是只是采用对您的情况有意义的任何原则和实践,您都应该能够将这些想法应用到这本书。如果您使用高质量的软件及时为企业提供价值,并且您的团队不断努力改进,您会在这里找到有用的信息。同时,我们认为某些特定的敏捷实践对任何团队的成功都至关重要。我们将在整本书中讨论这些。
We use the word “agile” in this book in a broad sense. Whether your team is practicing a particular agile method, such as Scrum, XP, Crystal, DSDM, or FDD, to name a few, or just adopting whatever principles and practices make sense for your situation, you should be able to apply the ideas in this book. If you’re delivering value to the business in a timely manner with high-quality software, and your team continually strives to improve, you’ll find useful information here. At the same time, there are particular agile practices we feel are crucial to any team’s success. We’ll talk about these throughout the book.
第 21 章,“关键成功因素”,列出了敏捷测试的关键成功因素。
Chapter 21, “Key Success Factors,” lists key success factors for agile testing.
您可能已经注意到,我们使用术语“测试人员”来描述主要活动围绕测试和质量保证展开的人员。您还会看到,我们经常使用“程序员”一词来描述主要活动围绕编写生产代码的人。我们并不打算让这些术语听起来狭隘或无关紧要。程序员所做的不仅仅是将规范转化为程序。我们不称他们为“开发人员”,因为每个人参与交付软件的是开发人员。测试人员所做的不仅仅是执行“测试任务”。每个敏捷团队成员都专注于交付提供商业价值的高质量产品。敏捷测试人员致力于确保他们的团队交付客户所需的质量。为方便起见,我们使用术语“程序员”和“测试人员”。
You might have noticed that we use the term “tester” to describe a person whose main activities revolve around testing and quality assurance. You’ll also see that we often use the word “programmer” to describe a person whose main activities revolve around writing production code. We don’t intend that these terms sound narrow or insignificant. Programmers do more than turn a specification into a program. We don’t call them “developers,” because everyone involved in delivering software is a developer. Testers do more than perform “testing tasks.” Each agile team member is focused on delivering a high-quality product that provides business value. Agile testers work to ensure that their team delivers the quality their customers need. We use the terms “programmer” and “tester” for convenience.
敏捷团队使用的几个核心实践与测试有关。敏捷程序员使用测试驱动开发 (TDD)(也称为测试驱动设计)来编写高质量的生产代码。使用 TDD,程序员为一小部分功能编写测试,看到它失败,编写使其通过的代码,然后继续下一个小功能。程序员还编写代码集成测试以确保小代码单元按预期协同工作。这一基本实践已被许多团队采用,甚至是那些不称自己为“敏捷”的团队,因为它只是一种思考软件设计和防止缺陷的聪明方法。图 1-2显示了程序员可能会看到的示例单元测试结果。
Several core practices used by agile teams relate to testing. Agile programmers use test-driven development (TDD), also called test-driven design, to write quality production code. With TDD, the programmer writes a test for a tiny bit of functionality, sees it fail, writes the code that makes it pass, and then moves on to the next tiny bit of functionality. Programmers also write code integration tests to make sure the small units of code work together as intended. This essential practice has been adopted by many teams, even those that don’t call themselves “agile,” because it’s just a smart way to think through your software design and prevent defects. Figure 1-2 shows a sample unit test result that a programmer might see.
本书与单元级或组件级测试无关,但这些类型的测试对于成功的项目至关重要。Brian Marick [2003] 将这些类型的测试描述为“支持团队”,帮助程序员知道接下来要编写什么代码。Brian 还创造了术语“面向技术的测试”,即属于程序员领域并使用程序员术语和行话描述的测试。在第二部分中,我们介绍了敏捷测试象限并检查了敏捷测试的不同类别。如果您想了解有关编写单元和组件测试以及 TDD 的更多信息,参考书目将引导您找到一些不错的资源。
This book isn’t about unit-level or component-level testing, but these types of tests are critical to a successful project. Brian Marick [2003] describes these types of tests as “supporting the team,” helping the programmers know what code to write next. Brian also coined the term “technology-facing tests,” tests that fall into the programmer’s domain and are described using programmer terms and jargon. In Part II, we introduce the Agile Testing Quadrants and examine the different categories of agile testing. If you want to learn more about writing unit and component tests, and TDD, the bibliography will steer you to some good resources.
如果您想了解应用于测试的敏捷价值观、原则和实践如何帮助您作为测试人员做到最好,并帮助您的团队交付更多业务价值,请继续阅读。如果你费心拿起这本书,你可能是那种不断努力成长和学习的专业人士。您可能拥有优秀的敏捷团队成功所需的心态。本书将向您展示改进组织产品、为团队提供最大价值并享受工作的方法。
If you want to know how agile values, principles, and practices applied to testing can help you, as a tester, do your best work, and help your team deliver more business value, please keep reading. If you’ve bothered to pick up this book, you’re probably the kind of professional who continually strives to grow and learn. You’re likely to have the mind-set that a good agile team needs to succeed. This book will show you ways to improve your organization’s product, provide the most value possible to your team, and enjoy your job.
如果您在软件行业工作了很长时间,您可能有机会感觉自己像 Lisa 的朋友。当您的任务无法完成时,更努力和更长时间的工作无济于事。敏捷开发承认这样一个现实,即我们在一天或一周内只有那么多的高效工作时间,并且我们无法计划改变的不可避免性。
If you’ve worked in the software industry long, you’ve probably had the opportunity to feel like Lisa’s friend. Working harder and longer doesn’t help when your task is impossible to achieve. Agile development acknowledges the reality that we only have so many good productive hours in a day or week, and that we can’t plan away the inevitability of change.
敏捷开发鼓励我们作为一个团队来解决我们的问题。业务人员、程序员、测试人员、分析师——参与软件开发的每个人——共同决定如何最好地改进他们的产品。最重要的是,作为测试人员,我们与一群人一起工作,他们都认为有责任提供尽可能最好的质量,并且都专注于测试。我们喜欢做这项工作,您也会喜欢。
Agile development encourages us to solve our problems as a team. Business people, programmers, testers, analysts—everyone involved in software development—decides together how best to improve their product. Best of all, as testers, we’re working together with a team of people who all feel responsible for delivering the best possible quality, and who are all focused on testing. We love doing this work, and you will too.
当我们在本书中提到“敏捷测试”时,我们通常是在谈论面向业务的测试,即定义业务专家所需特性和功能的测试。我们认为“面向客户”是“面向业务”的同义词。本书中的“测试”还包括批评产品的测试,并专注于发现成品中可能缺少的东西,以便我们改进它。它包括单元和组件级别测试之外的几乎所有内容:功能、系统、负载、性能、安全性、压力、可用性、探索性、端到端和用户接受度。所有这些类型的测试都可能适用于任何给定的项目,无论是敏捷项目还是使用更传统方法的项目。
When we say “agile testing” in this book, we’re usually talking about business-facing tests, tests that define the business experts’ desired features and functionality. We consider “customer-facing” a synonym for “business-facing.” “Testing” in this book also includes tests that critique the product and focus on discovering what might be lacking in the finished product so that we can improve it. It includes just about everything beyond unit and component level testing: functional, system, load, performance, security, stress, usability, exploratory, end-to-end, and user acceptance. All these types of tests might be appropriate to any given project, whether it’s an agile project or one using more traditional methodologies.
敏捷测试不仅仅意味着在敏捷项目上进行测试。一些测试方法,例如探索性测试,本质上是敏捷的,无论它是否完成了一个敏捷项目。测试一个应用程序并计划在你进行时了解它,并让这些信息指导你的测试,这与评估工作软件和响应变化是一致的。后面的章节讨论敏捷测试形式以及“敏捷测试”实践。
Agile testing doesn’t just mean testing on an agile project. Some testing approaches, such as exploratory testing, are inherently agile, whether it’s done an agile project or not. Testing an application with a plan to learn about it as you go, and letting that information guide your testing, is in line with valuing working software and responding to change. Later chapters discuss agile forms of testing as well as “agile testing” practices.
我们将在本书中大量讨论“客户团队”和“开发团队”。它们之间的区别在于它们在交付产品时所具备的技能。
We’ll talk a lot in this book about the “customer team” and the “developer team.” The difference between them is the skills they bring to delivering a product.
客户团队包括业务专家、产品所有者、领域专家、产品经理、业务分析师、主题专家——项目“业务”方面的每个人。客户团队编写开发团队交付的故事或功能集。他们提供的示例将以面向业务的测试形式推动编码。他们在每次迭代中与开发团队沟通和协作,回答问题,在白板上绘制示例,以及审查完成的故事或故事的一部分。
The customer team includes business experts, product owners, domain experts, product managers, business analysts, subject matter experts—everyone on the “business” side of a project. The customer team writes the stories or feature sets that the developer team delivers. They provide the examples that will drive coding in the form of business-facing tests. They communicate and collaborate with the developer team throughout each iteration, answering questions, drawing examples on the whiteboard, and reviewing finished stories or parts of stories.
测试人员是客户团队不可或缺的成员,帮助引出需求和示例,并帮助客户将他们的需求表达为测试。
Testers are integral members of the customer team, helping elicit requirements and examples and helping the customers express their requirements as tests.
参与交付代码的每个人都是开发人员,并且是开发团队的一部分。敏捷原则鼓励团队成员进行多项活动;任何团队成员都可以承担任何类型的任务。许多敏捷实践者不鼓励团队中的专门角色,并鼓励所有团队成员尽可能地将他们的技能传授给其他人。然而,每个团队都需要决定他们的项目需要什么专业知识。程序员、系统管理员、架构师、数据库管理员、技术作家、安全专家以及身兼多职的人员可能是团队的一部分,无论是物理上还是虚拟上。
Everyone involved with delivering code is a developer, and is part of the developer team. Agile principles encourage team members to take on multiple activities; any team member can take on any type of task. Many agile practitioners discourage specialized roles on teams and encourage all team members to transfer their skills to others as much as possible. Nevertheless, each team needs to decide what expertise their projects require. Programmers, system administrators, architects, database administrators, technical writers, security specialists, and people who wear more than one of these hats might be part of the team, physically or virtually.
测试人员也在开发团队中,因为测试是敏捷软件开发的核心组成部分。测试人员代表客户提倡质量,并协助开发团队实现最大的商业价值。
Testers are also on the developer team, because testing is a central component of agile software development. Testers advocate for quality on behalf of the customer and assist the development team in delivering the maximum business value.
客户和开发团队始终紧密合作。理想情况下,他们只是一个拥有共同目标的团队。该目标是为组织提供价值。敏捷项目在迭代中取得进展,迭代是通常持续一到四个星期的小开发周期。客户团队在开发人员的输入下,将确定要开发的故事的优先级,而开发团队将确定他们可以承担多少工作。他们将一起工作,通过测试和示例定义需求,并编写使测试通过的代码。测试人员涉足各个领域,了解客户的观点以及技术实施的复杂性(见图1-3)。
The customer and developer teams work closely together at all times. Ideally, they’re just one team with a common goal. That goal is to deliver value to the organization. Agile projects progress in iterations, which are small development cycles that typically last from one to four weeks. The customer team, with input from the developers, will prioritize stories to be developed, and the developer team will determine how much work they can take on. They’ll work together to define requirements with tests and examples, and write the code that makes the tests pass. Testers have a foot in each world, understanding the customer viewpoint as well as the complexities of the technical implementation (see Figure 1-3).
一些敏捷团队没有任何成员将自己定义为“测试人员”。然而,他们都需要有人帮助客户团队为迭代的故事编写面向业务的测试,确保测试通过,并确保足够的回归测试是自动化的。即使一个团队确实有测试人员,整个敏捷团队也要负责这些测试任务。我们在敏捷团队中的经验表明,测试技能和经验对于项目成功至关重要,并且测试人员确实为敏捷团队增加了价值。
Some agile teams don’t have any members who define themselves as “testers.” However, they all need someone to help the customer team write business-facing tests for the iteration’s stories, make sure the tests pass, and make sure that adequate regression tests are automated. Even if a team does have testers, the entire agile team is responsible for these testing tasks. Our experience with agile teams has shown that testing skills and experience are vital to project success and that testers do add value to agile teams.
我们都在世纪之交开始在敏捷团队中工作。与许多刚接触敏捷的测试人员一样,我们一开始并不知道会发生什么。我们与各自的敏捷团队一起工作,我们学到了很多关于敏捷项目测试的知识。我们还实施了其他敏捷测试人员和团队建议的想法和实践。多年来,我们也与其他敏捷测试人员分享了我们的经验。我们在敏捷和测试会议上促进了研讨会和指导教程,与本地用户组交谈,并加入了无数关于敏捷测试邮件列表的讨论。通过这些经验,我们确定了敏捷团队测试与传统瀑布式开发项目测试之间的差异。敏捷开发在很多方面改变了测试行业。
We both started working on agile teams at the turn of the millennium. Like a lot of testers who are new to agile, we didn’t know what to expect at first. Together with our respective agile teams, we’ve worked on we’ve learned a lot about testing on agile projects. We’ve also implemented ideas and practices suggested by other agile testers and teams. Over the years, we’ve shared our experiences with other agile testers as well. We’ve facilitated workshops and led tutorials at agile and testing conferences, talked with local user groups, and joined countless discussions on agile testing mailing lists. Through these experiences, we’ve identified differences between testing on agile teams and testing on traditional waterfall development projects. Agile development has transformed the testing profession in many ways.
与程序员密切合作或从最早阶段参与项目对我们来说都不是新鲜事。然而,我们习惯于严格执行狭义软件开发生命周期的门控阶段,从发布计划和需求定义开始,通常以匆忙的测试阶段和延迟发布结束。事实上,我们经常被推为看门人的角色,告诉业务经理,“抱歉,要求已冻结;我们可以在下一个版本中添加该功能。”
Neither working closely with programmers nor getting involved with a project from the earliest phases was new to us. However, we were used to strictly enforced gated phases of a narrowly defined software development life cycle, starting with release planning and requirements definition and usually ending with a rushed testing phase and a delayed release. In fact, we often were thrust into a gatekeeper role, telling business managers, “Sorry, the requirements are frozen; we can add that feature in the next release.”
作为质量保证团队的领导者,我们也经常被期望充当质量的守门人。我们无法控制代码的编写方式,甚至无法控制是否有任何程序员测试了他们的代码,除非通过我们个人的协作努力。我们的开发后测试阶段有望在代码完成后提高质量。我们有控制的错觉。我们通常拥有生产的关键,有时我们有权推迟发布或阻止它们继续进行。Lisa 甚至拥有“质量负责人”的称号,但实际上她只是 QA 团队的经理。
As leaders of quality assurance teams, we were also often expected to act as gatekeepers of quality. We couldn’t control how the code was written, or even if any programmers tested their code, other than by our personal efforts at collaboration. Our post-development testing phases were expected to boost quality after code was complete. We had the illusion of control. We usually had the keys to production, and sometimes we had the power to postpone releases or stop them from going forward. Lisa even had the title of “Quality Boss,” when in fact she was merely the manager of the QA team.
我们的开发周期通常很长。生产数据库软件的公司的项目可能会持续一年。Lisa 在一家互联网初创公司经历的六个月发布周期在当时看来很短,尽管冻结需求仍然是一段很长的时间。尽管有很多过程和纪律,在进入下一阶段之前努力完成一个阶段,但竞争有足够的时间领先,而且应用程序并不总是客户所期望的。
Our development cycles were generally long. Projects at a company that produced database software might last for a year. The six-month release cycles Lisa experienced at an Internet start-up seemed short at the time, although it was still a long time to have frozen requirements. In spite of much process and discipline, diligently completing one phase before moving on to the next, it was plenty of time for the competition to come out ahead, and the applications were not always what the customers expected.
传统团队专注于确保所有指定要求都在最终产品中交付。如果在最初的目标发布日期之前一切还没有准备好,发布通常会被推迟。开发团队通常不知道发布中有哪些功能,或者它们应该如何工作。个别程序员倾向于专注于代码的特定区域。测试人员研究需求文档来编写他们的测试计划,然后等待工作交付给他们进行测试。
Traditional teams are focused on making sure all the specified requirements are delivered in the final product. If everything isn’t ready by the original target release date, the release is usually postponed. The development teams don’t usually have input about what features are in the release, or how they should work. Individual programmers tend to specialize in a particular area of the code. Testers study the requirements documents to write their test plans, and then they wait for work to be delivered to them for testing.
过渡到敏捷项目的短迭代可能会产生最初的震惊和敬畏。我们怎么可能定义需求,然后在一、二、三或四个星期内测试和交付生产就绪代码?这对于拥有不同职能的独立团队的大型组织来说尤其困难,对于地理位置分散的团队来说更难。所有这些不同的程序员、测试人员、分析师、项目经理和无数专业人士在新的敏捷项目中处于什么位置?我们怎么可能这么快地编码和测试呢?我们在哪里可以找到时间来进行自动化测试等困难的工作?我们对交付到生产环境的错误代码有什么控制?
Transitioning to the short iterations of an agile project might produce initial shock and awe. How can we possibly define requirements and then test and deliver production-ready code in one, two, three, or four weeks? This is particularly tough for larger organizations with separate teams for different functions and even harder for teams that are geographically dispersed. Where do all these various programmers, testers, analysts, project managers, and countless specialties fit in a new agile project? How can we possibly code and test so quickly? Where would we find time for difficult efforts such as automating tests? What control do we have over bad code getting delivered to production?
我们将分享我们第一次敏捷体验的故事,向您展示每个人都必须从某个地方开始。
We’ll share our stories from our first agile experiences to show you that everyone has to start somewhere.
敏捷团队与业务紧密合作,对需求有详细的了解。他们专注于他们可以提供的价值,并且他们可能对确定功能的优先级有大量的投入。测试人员不会坐等工作;他们站起来寻找在整个开发周期及以后做出贡献的方法。
Agile teams work closely with the business and have a detailed understanding of the requirements. They’re focused on the value they can deliver, and they might have a great deal of input into prioritizing features. Testers don’t sit and wait for work; they get up and look for ways to contribute throughout the development cycle and beyond.
如果对敏捷项目的测试感觉就像对传统项目的测试一样,我们就不会觉得有必要写一本书了。让我们比较和对比这些测试方法。
If testing on an agile project felt just like testing on a traditional project, we wouldn’t feel the need to write a book. Let’s compare and contrast these testing methods.
它有助于从查看敏捷测试与传统软件开发中的测试之间的相似性开始。考虑图 1-4。
It helps to start by looking at similarities between agile testing and testing in traditional software development. Consider Figure 1-4.
在分阶段方法图中,很明显测试发生在最后,就在发布之前。该图是理想化的,因为它给人的印象是,用于测试的时间与用于编码的时间一样多。在许多项目中,情况并非如此。测试被“压缩”了,因为编码花费的时间比预期的要长,而且团队最终进入了编码和修复的循环。
In the phased approach diagram, it is clear that testing happens at the end, right before release. The diagram is idealistic, because it gives the impression there is as much time for testing as there is for coding. In many projects, this is not the case. The testing gets “squished” because coding takes longer than expected, and because teams get into a code-and-fix cycle at the end.
敏捷是迭代和增量的。这意味着测试人员会在编码完成后立即测试每个编码增量。一次迭代可能短至一周,也可能长至一个月。该团队构建并测试了一些代码,确保其正常工作,然后继续进行下一个需要构建的部分。程序员永远不会领先于测试人员,因为一个故事只有经过测试才算“完成”。我们将在整本书中更多地讨论这一点。
Agile is iterative and incremental. This means that the testers test each increment of coding as soon as it is finished. An iteration might be as short as one week, or as long as a month. The team builds and tests a little bit of code, making sure it works correctly, and then moves on to next piece that needs to be built. Programmers never get ahead of the testers, because a story is not “done” until it has been tested. We’ll talk much more about this throughout the book.
敏捷团队采用的项目方法多种多样。一个团队可能专注于一个项目,也可能是另一个项目的一部分更大的项目。无论您的项目有多大,您仍然需要从某个地方开始。您的团队可能会在估算会议上处理一个史诗或功能,一组相关的故事,或者您可能会开会计划发布。无论项目或项目的子集是如何开始的,您都需要对其有一个高层次的理解。在准备发布时,您可能会想出一个测试计划或策略,但它可能看起来与您之前完成的任何测试计划都大不相同。
There’s tremendous variety in the approaches to projects that agile teams take. One team might be dedicated to a single project or might be part of another bigger project. No matter how big your project is, you still have to start somewhere. Your team might take on an epic or feature, a set of related stories at an estimating meeting, or you might meet to plan the release. Regardless of how a project or subset of a project gets started, you’ll need to get a high-level understanding of it. You might come up with a plan or strategy for testing as you prepare for a release, but it will probably look quite different from any test plan you’ve done before.
每个项目、每个团队,有时甚至每个迭代都是不同的。你的团队如何解决问题应该取决于问题、人员和你可用的工具。作为敏捷团队成员,您需要适应团队的需求。
Every project, every team, and sometimes every iteration is different. How your team solves problems should depend on the problem, the people, and the tools you have available. As an agile team member, you will need to be adaptive to the team’s needs.
与其根据业务分析师在任何人想到编写一行代码之前创建的需求文档创建测试,不如有人需要在编码开始前几天或几小时编写测试来说明每个故事的需求。这通常是一个团队之间的协作努力业务或领域专家以及测试人员、分析师或其他开发团队成员。详细的功能测试用例,最好基于业务专家提供的示例,充实需求。测试人员将进行手动探索性测试,以发现定义的测试用例可能遗漏的重要错误。随着对每个故事的编码进行,测试人员可能会与其他开发人员配对以自动化和执行测试用例。自动功能测试被添加到回归测试套件中。当演示最小功能的测试完成时,团队可以认为故事已经完成。
Rather than creating tests from a requirements document that was created by business analysts before anyone ever thought of writing a line of code, someone will need to write tests that illustrate the requirements for each story days or hours before coding begins. This is often a collaborative effort between a business or domain expert and a tester, analyst, or some other development team member. Detailed functional test cases, ideally based on examples provided by business experts, flesh out the requirements. Testers will conduct manual exploratory testing to find important bugs that defined test cases might miss. Testers might pair with other developers to automate and execute test cases as coding on each story proceeds. Automated functional tests are added to the regression test suite. When tests demonstrating minimum functionality are complete, the team can consider the story finished.
如果你参加过本世纪初的敏捷会议和研讨会,你会听到很多关于 TDD 和验收测试的信息,但很少听到其他关键类型的测试,例如负载、性能、安全性、可用性和其他“能力”测试。作为测试人员,我们认为这有点奇怪,因为所有这些类型的测试对于敏捷项目和使用任何其他开发方法的项目一样重要。真正的区别在于,我们喜欢在开发过程中尽早进行这些测试,以便它们也可以驱动设计和编码。
If you attended agile conferences and seminars in the early part of this decade, you heard a lot about TDD and acceptance testing but not so much about other critical types of testing, such as load, performance, security, usability, and other “ility” testing. As testers, we thought that was a little weird, because all these types of testing are just as vital on agile projects as they are on projects using any other development methodology. The real difference is that we like to do these tests as early in the development process as we can so that they can also drive design and coding.
如果团队像 Lisa 的团队那样实际发布每个迭代,则每个迭代的最后一两天就是“结束游戏”,此时可以进行用户验收测试、培训、错误修复和部署到暂存环境。其他团队,例如 Janet 的团队,每隔几次迭代就发布一次,甚至可能有整个迭代的“结束游戏”活动来验证发布准备情况。这里的区别是所有的测试都没有留到最后。
If the team actually releases each iteration, as Lisa’s team does, the last day or two of each iteration is the “end game,” the time when user acceptance testing, training, bug fixing, and deployments to staging environments can occur. Other teams, such as Janet’s, release every few iterations, and might even have an entire iteration’s worth of “end game” activities to verify release readiness. The difference here is that all the testing is not left until the end.
作为敏捷团队的测试人员,您是将代码发布到生产环境的关键角色,就像您在更传统的环境中一样。您可能会运行脚本或进行手动测试以验证发布的所有元素(例如数据库更新脚本)是否就位。所有团队成员都参与可能针对每次迭代或每次发布进行的回顾或其他流程改进活动。整个团队集思广益解决问题并改进流程和实践的方法。
As a tester on an agile team, you’re a key player in releasing code to production, just as you might have been in a more traditional environment. You might run scripts or do manual testing to verify all elements of a release, such as database update scripts, are in place. All team members participate in retrospectives or other process improvement activities that might occur for every iteration or every release. The whole team brainstorms ways to solve problems and improve processes and practices.
敏捷项目有多种风格。您的团队是否在全新的(新)开发项目中从头开始?如果是这样,与在没有自动回归套件的遗留系统上重写或构建的团队相比,您面临的挑战可能更少。与第三方合作会给任何团队带来额外的测试挑战。
Agile projects have a variety of flavors. Is your team starting with a clean slate, in a greenfield (new) development project? If so, you might have fewer challenges than a team faced with rewriting or building on a legacy system that has no automated regression suite. Working with a third party brings additional testing challenges to any team.
无论您使用哪种开发方式,软件开发生命周期中几乎相同的元素都需要发生。区别敏捷的特点是时间框架大大缩短,活动同时发生。参与者、测试和工具需要具有适应性。
Whatever flavor of development you’re using, pretty much the same elements of a software development life cycle need to happen. The difference with agile is that time frames are greatly shortened, and activities happen concurrently. Participants, tests, and tools need to be adaptive.
敏捷项目中测试人员最关键的区别是来自测试的快速反馈。它推动项目向前发展,如果未达到某些里程碑,没有守门人准备阻止项目进展。
The most critical difference for testers in an agile project is the quick feedback from testing. It drives the project forward, and there are no gatekeepers ready to block project progress if certain milestones aren’t met.
我们遇到过抵制向敏捷开发过渡的测试人员,他们担心“敏捷开发”等同于混乱、缺乏纪律、缺乏文档以及对测试人员不利的环境。虽然一些团队似乎确实使用“敏捷”这个流行语来证明他们想做什么就做什么,但真正的敏捷团队都是关于可重复的质量和效率。根据我们的经验,敏捷团队是测试人员的绝佳去处。
We’ve encountered testers who resist the transition to agile development, fearing that “agile development” equates with chaos, lack of discipline, lack of documentation, and an environment that is hostile to testers. While some teams do seem to use the “agile” buzzword to justify simply doing whatever they want, true agile teams are all about repeatable quality as well as efficiency. In our experience, an agile team is a wonderful place to be a tester.
敏捷开发与传统开发的最大区别之一是敏捷的“全团队”方法。有了敏捷,感觉对质量负责的不仅仅是测试人员或质量保证团队。我们不考虑“部门”,我们只考虑交付最佳产品所需的技能和资源。敏捷开发的重点是在一个时间框架内生产高质量的软件,使其对业务的价值最大化。这是整个团队的工作,而不仅仅是测试人员或指定的质量保证专业人员。敏捷团队中的每个人都会“被测试感染”。测试,从单元级别开始,推动编码,帮助团队了解应用程序应该如何工作,并让我们知道我们何时“完成”任务或故事。
One of the biggest differences in agile development versus traditional development is the agile “whole-team” approach. With agile, it’s not only the testers or a quality assurance team who feel responsible for quality. We don’t think of “departments,” we just think of the skills and resources we need to deliver the best possible product. The focus of agile development is producing high-quality software in a time frame that maximizes its value to the business. This is the job of the whole team, not just testers or designated quality assurance professionals. Everyone on an agile team gets “test-infected.” Tests, from the unit level on up, drive the coding, help the team learn how the application should work, and let us know when we’re “done” with a task or story.
敏捷团队必须具备生成高质量代码所需的所有技能,这些代码可提供组织所需的功能。虽然这可能意味着在团队中包括专家,例如专家测试人员,但它并不将特定任务限制为特定团队成员。任何任务都可能由任何团队成员或一对团队成员完成。这意味着团队负责各种测试任务,例如自动化测试和手动探索性测试。这也意味着整个团队不断地考虑设计可测试性代码。
An agile team must possess all the skills needed to produce quality code that delivers the features required by the organization. While this might mean including specialists on the team, such as expert testers, it doesn’t limit particular tasks to particular team members. Any task might be completed by any team member, or a pair of team members. This means that the team takes responsibility for all kinds of testing tasks, such as automating tests and manual exploratory testing. It also means that the whole team thinks constantly about designing code for testability.
全团队方法涉及持续协作。测试人员与程序员、客户团队和其他团队专家协作——不仅是为了测试任务,还有与测试相关的其他任务,例如构建基础设施和设计可测试性。图 1-5显示了开发人员与两个客户和一个测试人员(未图示)一起审查报告。
The whole-team approach involves constant collaboration. Testers collaborate with programmers, the customer team, and other team specialists—and not just for testing tasks, but other tasks related to testing, such as building infrastructure and designing for testability. Figure 1-5 shows a developer reviewing reports with two customers and a tester (not pictured).
全团队方法意味着每个人都对测试任务负责。这意味着团队成员拥有一系列技能和经验,可用于应对挑战,例如通过将示例转化为测试并转化为代码以使这些测试通过的可测试性设计。这些不同的观点只能意味着更好的测试和测试覆盖率。
The whole-team approach means everyone takes responsibility for testing tasks. It means team members have a range of skill sets and experience to employ in attacking challenges such as designing for testability by turning examples into tests and into code to make those tests pass. These diverse viewpoints can only mean better tests and test coverage.
最重要的是,在敏捷团队中,任何人都可以寻求和获得帮助。团队致力于提供尽可能高的团队商业价值,并且团队会尽一切努力来实现它。一些刚接触敏捷的人认为敏捷就是速度。事实是,一切都与质量有关——如果不是,我们会质疑它是否真的是一个“敏捷”团队。
Most importantly, on an agile team, anyone can ask for and receive help. The team commits to providing the highest possible business value as a team, and the team does whatever is needed to deliver it. Some folks who are new to agile perceive it as all about speed. The fact is, it’s all about quality—and if it’s not, we question whether it’s really an “agile” team.
你的情况很独特。这就是为什么您需要了解您的团队可能面临的潜在测试障碍以及如何应用敏捷价值观和原则来克服这些障碍。
Your situation is unique. That’s why you need to be aware of the potential testing obstacles your team might face and how you can apply agile values and principles to overcome them.
了解测试人员在敏捷团队中执行的活动有助于您向自己的团队展示测试人员可以增加的价值。学习敏捷测试的核心实践将帮助您的团队交付令客户满意的软件。
Understanding the activities that testers perform on agile teams helps you show your own team the value that testers can add. Learning the core practices of agile testing will help your team deliver software that delights your customers.
在本章中,我们解释了使用术语“敏捷测试”时的含义。
In this chapter, we’ve explained what we mean when we use the term “agile testing.
我们展示了敏捷宣言如何与测试相关,它强调个人和交互、工作软件、客户协作和响应变化。
We showed how the Agile Manifesto relates to testing, with its emphasis on individuals and interactions, working software, customer collaboration, and responding to change.
我们为本书提供了一些背景信息,包括我们使用的一些其他术语,例如“测试人员”、“程序员”、“客户”和相关术语,以便我们可以使用共同语言。
We provided some context for this book, including some other terms we use such as “tester,” “programmer,” “customer,” and related terms so that we can speak a common language.
我们解释了专注于业务价值和交付客户所需质量的敏捷测试与专注于需求一致性的传统测试有何不同。
We explained how agile testing, with its focus on business value and delivering the quality customers require, is different from traditional testing, which focuses on conformance to requirements.
我们引入了“全团队”敏捷测试方法,这意味着参与交付软件的每个人都有责任交付高质量的软件。
We introduced the “whole-team” approach to agile testing, which means that everyone involved with delivering software is responsible for delivering high-quality software.
我们建议采取实用的方法,通过应用敏捷价值观和原则来克服在您的独特情况下出现的敏捷测试障碍。
We advised taking a practical approach by applying agile values and principles to overcome agile testing obstacles that arise in your unique situation.
敏捷团队中的每个人都是测试人员。任何人都可以接手测试任务。如果那是真的,那么敏捷测试人员有什么特别之处呢?如果我将自己定义为敏捷团队的测试员,那究竟意味着什么?敏捷测试人员是否需要与传统团队中的测试人员不同的技能组合?是什么指导他们进行日常活动?
Everyone on an agile team is a tester. Anyone can pick up testing tasks. If that’s true, then what is special about an agile tester? If I define myself as a tester on an agile team, what does that really mean? Do agile testers need different skill sets than testers on traditional teams? What guides them in their daily activities?
在本章中,我们将讨论敏捷测试思维模式,展示敏捷价值观和原则如何指导测试,并概述测试人员如何为敏捷团队增加价值。
In this chapter, we talk about the agile testing mind-set, show how agile values and principles guide testing, and give an overview of how testers add value on agile teams.
我们这样定义敏捷测试人员:拥抱变化的专业测试人员,与技术人员和业务人员合作良好,理解使用测试记录需求和推动开发的概念。敏捷测试人员往往具有良好的技术技能,知道如何与其他人进行自动化测试,并且也是经验丰富的探索性测试人员。他们愿意了解客户的行为,以便更好地了解客户的软件需求。
We define an agile tester this way: a professional tester who embraces change, collaborates well with both technical and business people, and understands the concept of using tests to document requirements and drive development. Agile testers tend to have good technical skills, know how to collaborate with others to automate tests, and are also experienced exploratory testers. They’re willing to learn what customers do so that they can better understand the customers’ software requirements.
谁是敏捷测试员?她是推动敏捷测试的团队成员。我们认识许多敏捷测试人员,他们都是从其他专业开始的。开发人员被测试感染并扩展到单元测试之外。习惯于以敏捷方式工作的探索性测试人员被敏捷团队的想法所吸引。担任其他角色的专业人员,例如业务或职能分析师,可能具有相同的特征并做很多相同的工作。
Who’s an agile tester? She’s a team member who drives agile testing. We know many agile testers who started out in some other specialization. A developer becomes test-infected and branches out beyond unit testing. An exploratory tester, accustomed to working in an agile manner, is attracted to the idea of an agile team. Professionals in other roles, such as business or functional analysts, might share the same traits and do much of the same work.
能力固然重要,但态度更重要。珍妮特喜欢说:“没有态度,技巧就一文不值。” 在不得不为我们的敏捷团队雇佣大量测试人员之后,我们对此进行了很多思考,并与敏捷社区中的其他人进行了讨论。测试人员倾向于看到全局。他们更多地从用户或客户的角度来看待应用程序,这意味着他们通常以客户为中心。
Skills are important, but attitude counts more. Janet likes to say, “Without the attitude, the skill is nothing.” Having had to hire numerous testers for our agile teams, we’ve put a lot of thought into this and discussed it with others in the agile community. Testers tend to see the big picture. They look at the application more from a user or customer point of view, which means they’re generally customer-focused.
是什么让团队“敏捷”?对我们来说,敏捷团队是一个不断专注于尽最大努力并交付尽可能最好的产品的团队。根据我们的经验,这涉及大量的纪律、学习、时间、实验和合作。它并不适合所有人,但它非常适合我们这些喜欢团队活力并专注于持续改进的人。
What makes a team “agile”? To us, an agile team is one that continually focuses on doing its best work and delivering the best possible product. In our experience, this involves a ton of discipline, learning, time, experimentation, and working together. It’s not for everyone, but it’s ideal for those of us who like the team dynamic and focus on continual improvement.
成功的项目是允许好人做好工作的结果。使某人在敏捷团队中成为成功的测试员的特征可能与在任何团队中成为高价值测试员的特征相同。
Successful projects are a result of good people allowed to do good work. The characteristics that make someone succeed as a tester on an agile team are probably the same characteristics that make a highly valued tester on any team.
敏捷测试人员不会将自己视为质量警察,保护她的客户免受不适当代码的影响。她准备好收集和共享信息,与客户或产品负责人一起工作,以帮助他们充分表达他们的需求,以便他们能够获得所需的功能,并向每个人提供项目进展的反馈。
An agile tester doesn’t see herself as a quality police officer, protecting her customers from inadequate code. She’s ready to gather and share information, to work with the customer or product owner in order to help them express their requirements adequately so that they can get the features they need, and to provide feedback on project progress to everyone.
敏捷测试人员,也许还有任何具有正确技能和思维方式的测试人员,都在不断寻找团队可以更好地生产高质量软件的方法。在个人层面上,这可能意味着参加本地用户组会议或圆桌会议以了解其他团队在做什么。它也意味着尝试新工具来帮助团队更好地指定、执行和自动化客户需求作为测试。
Agile testers, and maybe any tester with the right skills and mind-set, are continually looking for ways the team can do a better job of producing high-quality software. On a personal level, that might mean attending local user group meetings or roundtables to find out what other teams are doing. It also means trying out new tools to help the team do a better job of specifying, executing, and automating customer requirements as tests.
底线是敏捷测试人员,就像他们的敏捷队友一样,喜欢学习新技能和接受新挑战,他们不会局限于只解决测试问题。这不仅仅是测试人员的特征;我们在所有敏捷团队成员中都看到了这一点。敏捷测试人员帮助开发人员和客户团队解决可能出现的任何类型的问题。测试人员可以提供信息,帮助团队回顾并了解哪些有效,哪些无效。
The bottom line is that agile testers, like their agile teammates, enjoy learning new skills and taking on new challenges, and they don’t limit themselves to solving only testing issues. This isn’t just a trait of testers; we see it in all agile team members. Agile testers help the developer and customer teams address any kind of issue that might arise. Testers can provide information that helps the team look back and learn what’s working and what isn’t.
创造力、对想法的开放性、愿意承担任何任务或角色、关注客户以及对大局的持续看法只是敏捷测试思维模式的一些组成部分。优秀的测试人员对软件可能在何处以及如何失败以及如何追踪失败有直觉和理解。
Creativity, openness to ideas, willingness to take on any task or role, focus on the customer, and a constant view of the big picture are just some components of the agile testing mind-set. Good testers have an instinct and understanding for where and how software might fail, and how to track down failures.
测试人员可能在测试方面具有特殊的专业知识和经验,但优秀的敏捷测试人员不怕跳入设计讨论,提出有助于提高可测试性或创建更优雅的解决方案的建议。敏捷测试思维模式是一种以结果为导向、工匠般的、协作的、渴望学习的、热衷于及时交付业务价值的思维模式。
Testers might have special expertise and experience in testing, but a good agile tester isn’t afraid to jump into a design discussion with suggestions that will help testability or create a more elegant solution. An agile testing mind-set is one that is results-oriented, craftsman-like, collaborative, eager to learn, and passionate about delivering business value in a timely manner.
个人可以对项目的成功产生重大影响。我们期望一个拥有更多经验和更高技能成员的团队能够胜过一个缺乏天赋的团队。但是一个团队不仅仅是它的单个成员。敏捷价值观和原则提倡关注参与项目的人员以及他们如何互动和沟通。一个以敏捷价值观和原则指导自己的团队将比一个由人才组成的低效团队具有更高的团队士气和更快的速度。
Individuals can have a big impact on a project’s success. We’d expect a team with more experienced and higher-skilled members to outperform a less talented team. But a team is more than just its individual members. Agile values and principles promote a focus on the people involved in a project and how they interact and communicate. A team that guides itself with agile values and principles will have higher team morale and better velocity than a poorly functioning team of talented individuals.
我们在第一章开头介绍的敏捷宣言中的四个价值陈述显示的是偏好,而不是最后通牒,并且没有说明做什么或不做什么。敏捷宣言还包括一系列定义我们如何进行软件开发的原则。我们的敏捷“测试”原则列表部分源自这些原则。因为我们都来自极限编程文化,所以我们采纳了它的许多价值观和基本原则。我们还纳入了对我们的团队有效的指导方针和原则。您的团队自己的价值观和原则将在您选择实践和决定您希望如何工作时指导您。
The four value statements in the Agile Manifesto, which we presented at the start of the first chapter, show preferences, not ultimatums, and make no statements about what to do or not to do. The Agile Manifesto also includes a list of principles that define how we approach software development. Our list of agile “testing” principles is partially derived from those principles. Because we both come from the Extreme Programming culture, we’ve adopted many of its values and underlying principles. We’ve also incorporated guidelines and principles that have worked for our teams. Your team’s own values and principles will guide you as you choose practices and make decisions about how you want to work.
The principles we think are important for an agile tester are:
Deliver Value to the Customer.
Enable Face-to-Face Communication.
有勇气。
保持简单。
Practice Continuous Improvement.
响应变化。
自组织。
以人为本。
享受。
鉴于测试驱动敏捷项目,反馈在任何敏捷团队中都起着重要作用也就不足为奇了。测试人员作为“信息提供者”的传统角色使她对敏捷团队具有内在的价值。敏捷测试人员最重要的贡献之一是帮助产品所有者或客户以示例和测试的形式阐明每个故事的需求。然后,测试人员与队友一起工作,将这些需求转化为可执行的测试。测试人员、程序员和其他团队成员尽早并经常运行这些测试,以便他们不断得到有意义的反馈的指导。我们将在本书中花大量时间解释如何做到这一点。
Given that tests drive agile projects, it’s no surprise that feedback plays a big part in any agile team. The tester’s traditional role of “information provider” makes her inherently valuable to an agile team. One of the agile tester’s most important contributions is helping the product owner or customer articulate requirements for each story in the form of examples and tests. The tester then works together with teammates to turn those requirements into executable tests. Testers, programmers, and other team members work to run these tests early and often so they’re continually guided by meaningful feedback. We’ll spend a lot of time in this book explaining ways to do this.
当团队遇到障碍时,反馈是帮助消除障碍的一种方式。我们是否提供了一个不能完全满足客户期望的用户界面?让我们写一张任务卡,提醒我们在下一个 UI 故事的纸质原型上与客户合作。
When the team encounters obstacles, feedback is one way to help remove them. Did we deliver a user interface that didn’t quite meet customer expectations? Let’s write a task card reminding us to collaborate with the customer on paper prototypes of the next UI story.
管理层是否担心工作的进展情况?显示每天编写、运行和通过的测试的大图表。显示大图功能覆盖范围,例如测试矩阵。难以使构建稳定?Lisa 的团队显示了距离标记要发布的版本还有多少天,以便让每个人都专注于及时完成故事。在这成为一种习惯之后,他们就不再需要视觉提示了。
Is management worried about how work is progressing? Display a big visible chart of tests written, run, and passing every day. Display big-picture functionality coverage such as test matrices. Having trouble getting the build stable? Lisa’s team displayed the number of days remaining until time to tag the build for release in order to keep everyone focused on finishing stories in time. After that became a habit, they didn’t need the visual cue anymore.
敏捷开发是关于在提供客户最近优先考虑的功能的小版本中交付价值。这通常意味着限制范围。很容易被客户吸引团队对酷功能的渴望。任何人都可以质疑这些添加,但测试人员通常会认识到对故事的影响,因为他们需要考虑测试的影响。
Agile development is about delivering value in small releases that provide exactly the functionality that the customer has most recently prioritized. This usually means limiting scope. It’s easy to get caught up in the customer team’s desire for cool features. Anyone can question these additions, but a tester often recognizes the impact to the story, because they need to think about the testing repercussions.
敏捷测试人员始终专注于大局。我们可以在此迭代中交付最关键的功能,并在以后添加。如果我们让新功能潜入,我们就有可能无法按时交付任何东西。如果我们过于关注边缘案例而错过了快乐道路上的核心功能,我们将无法提供业务所需的价值。
Agile testers stay focused on the big picture. We can deliver the most critical functionality in this iteration and add to it later. If we let new features creep in, we risk delivering nothing on time. If we get too caught up with edge cases and miss core functionality on the happy path, we won’t provide the value the business needs.
敏捷测试人员采用与 Lisa 的故事中确定的方法相同的方法。虽然我们的一项技能是识别“快乐路径”之外的测试用例,但我们仍然需要从确保快乐路径有效开始。我们可以为快乐路径自动化测试,并在以后添加负面和边界测试。始终考虑什么能为客户增加最大价值,并了解您的背景。如果应用程序是安全关键的,则绝对需要添加负面测试。在估算过程中需要考虑测试时间,以确保在迭代中分配足够的时间来交付“安全”功能。
Agile testers take the same approach as that identified in Lisa’s story. While one of our skills is to identify test cases beyond the “happy path,” we still need to start by making sure the happy path works. We can automate tests for the happy path, and add negative and boundary tests later. Always consider what adds the most value to the customer, and understand your context. If an application is safety-critical, adding negative tests is absolutely required. The testing time needs to be considered during the estimation process to make sure that enough time is allotted in the iteration to deliver a “safe” feature.
没有良好的沟通,任何团队都无法运作良好。今天,当这么多团队分布在多个地理位置时,沟通是甚至更重要,更具挑战。敏捷测试人员应该寻找促进沟通的独特方式。这是做好她工作的一个关键方面。
No team works well without good communication. Today, when so many teams are distributed in multiple geographical locations, communication is even more vital and more of a challenge. The agile tester should look for unique ways to facilitate communication. It is a critical aspect to doing her job well.
任何时候出现关于功能应该如何工作或界面应该是什么样子的问题时,测试人员都可以召集程序员和业务专家来讨论。测试人员永远不应该妨碍任何直接的客户与开发人员的沟通,但他们通常可以帮助确保沟通的发生。
Any time there is a question about how a feature should work or what an interface should look like, the tester can pull in a programmer and a business expert to talk about it. Testers should never get in the way of any direct customer-developer communication, but they can often help to make sure that communication happens.
敏捷测试人员从客户的角度看待每个故事或主题,但也了解与实现功能相关的技术方面和限制。他们可以帮助客户和开发人员实现共同语言。业务人员和软件人员经常说不同的语言。他们必须找到一些共同点才能成功合作。测试人员可以帮助他们开发一种共享语言、项目方言或团队行话。
Agile testers see each story or theme from the customer’s point of view but also understand technical aspects and limitations related to implementing features. They can help customers and developers achieve a common language. Business people and software people often speak different languages. They have to find some common ground in order to work together successfully. Testers can help them develop a shared language, a project dialect, or team jargon.
Brian Marick (2003) 建议我们使用示例来开发这种语言。当 Lisa 的团队在冲刺计划会议期间进行哲学讨论时,Lisa 向产品负责人询问示例或使用场景。测试人员可以通过更多示例鼓励白板讨论。这些有助于客户更清楚地设想他们的要求。它们还帮助开发人员生成设计良好的代码以满足这些要求。
Brian Marick (2003) recommends that we use examples to develop this language. When Lisa’s team digresses into a philosophical discussion during a sprint planning meeting, Lisa asks the product owner for an example or usage scenario. Testers can encourage whiteboard discussions to work through more examples. These help the customers envision their requirements more clearly. They also help the developers to produce well-designed code to meet those requirements.
面对面的交流无可替代。敏捷开发依赖于持续的协作。与其他敏捷团队成员一样,执行测试任务的人员将不断寻找客户和技术团队成员进行讨论和协作。当敏捷测试人员怀疑隐藏的假设或被误解的需求时,她会让客户和开发人员讨论它。如果不同建筑物或大陆的人们需要交谈,他们会寻找创造性的方式来取代面对面的实时对话。
Face-to-face communication has no substitute. Agile development depends on constant collaboration. Like other agile team members, the people doing testing tasks will continually seek out customer and technical team members to discuss and collaborate. When an agile tester suspects a hidden assumption or a misunderstood requirement, she’ll get a customer and a developer talking about it. If people in a different building or continent need to talk, they look for creative ways to replace face-to-face, real-time conversations.
勇气是 XP 中的核心价值,测试自动化和持续集成等实践让团队践行这一价值。开发人员有勇气进行更改和重构代码,因为他们拥有自动回归套件的安全网。在本节中,我们将讨论过渡到敏捷团队时所需的情感勇气。
Courage is a core value in XP, and practices such as test automation and continuous integration allow the team to practice this value. The developers have the courage to make changes and refactor the code because they have the safety net of an automated regression suite. In this section, we talk about the emotional courage that is needed when transitioning to an agile team.
您是否曾在一个组织中工作,测试人员被困在自己的孤岛中,无法与业务利益相关者或技术团队的其他成员交谈?虽然您可能会抓住机会加入协作敏捷环境,但您可能会觉得不得不去询问客户示例,或者要求程序员帮助自动化测试或在每日站立会议中提出障碍,这让您感到不自在。
Have you worked in an organization where testers were stuck in their own silo, unable to talk to either business stakeholders or other members of the technical team? While you might jump at the chance to join a collaborative agile environment, you might feel uncomfortable having to go ask the customer for examples, or ask a programmer to help automate a test or bring up a roadblock during the daily stand-up.
当您第一次加入敏捷团队时,或者当您当前的团队第一次过渡到敏捷开发时,感到恐惧并列出需要回答的问题列表是很正常的。这么短的时间,我们要如何完成每个故事的测试任务?测试将如何“跟上”开发?你怎么知道多少测试就足够了?或者,您可能是一名功能测试经理或质量流程经理,您不清楚该角色在敏捷团队中的位置,而且没有人知道答案。敏捷测试人员需要勇气来找到这些问题的答案,但也有其他需要勇气的原因。
When you first join an agile team, or when your current team firsts transitions to agile development, it’s normal to experience fear and have a list of questions that need to be answered. How in the world are we going to be able to complete testing tasks for each story in such a short time? How will testing “keep up” with development? How do you know how much testing is enough? Or maybe you’re a functional testing manager or a quality process manager and it’s not clear to you where that role fits on an agile team, and nobody has the answers. Agile testers need courage to find the answers to those questions, but there are other reasons as well for having courage.
我们需要勇气让自己失败,知道至少我们会很快失败并能够从失败中吸取教训。在我们因为没有获得稳定的构建而导致迭代失败后,我们将开始考虑确保它不会再次发生的方法。
We need courage to let ourselves fail, knowing that at least we’ll fail fast and be able to learn from that failure. After we’ve blown an iteration because we didn’t get a stable build, we’ll start thinking of ways to ensure it doesn’t happen again.
我们需要勇气允许别人犯错,因为这是吸取教训的唯一途径。
We need courage to allow others to make mistakes, because that’s the only way to learn the lesson.
我们需要勇气寻求帮助,尤其是当可以提供帮助的人看起来非常忙碌而且压力很大时。走出你的旧筒仓并加入一个团队来为成功或失败负责需要勇气。提出问题或指出您认为的缺陷需要勇气,即使在一个由敏捷价值观和原则支持的团队中也是如此。不要害怕!敏捷团队是开放的,通常接受新想法。
We need courage to ask for help, especially when the person who could provide that help looks pretty busy and stressed-out himself. Climbing out of your old silo and joining in a team responsibility for success or failure takes courage. Asking a question or pointing out what you think is a flaw requires courage, even in a team supported by agile values and principles. Don’t be afraid! Agile teams are open and generally accepting of new ideas.
Kent Beck 的Extreme Programming Explained建议我们做可能可行的最简单的事情。这并不意味着您尝试的第一件事实际上会起作用,但它应该很简单。
Kent Beck’s Extreme Programming Explained advised us to do the simplest thing that could possibly work. That doesn’t mean the first thing you try will actually work, but it ought to be simple.
敏捷测试人员及其团队面临的挑战不仅是要生成最简单的软件实现,还要采取简单的方法来确保软件满足客户需求。这并不意味着团队不应该花一些时间来分析主题和故事并思考适当的架构和设计。这确实意味着当他们的需求可能有点复杂并且更简单的解决方案将提供相同的价值时,团队可能需要将其推回到团队的业务方面。
Agile testers and their teams are challenged to not only produce the simplest possible software implementation but to take a simple approach to ensuring that software meets the customer requirements. This doesn’t mean that the team shouldn’t take some time to analyze themes and stories and think through the appropriate architecture and design. It does mean that the team might need to push back to the business side of the team when their requirements might be a bit elaborate and a simpler solution will deliver the same value.
我们中的一些人在软件组织工作,作为测试人员和质量保证人员,我们被要求制定质量标准。我们认为这是倒退的,因为这取决于客户团队来决定他们想要支付什么样的质量水平。测试人员和其他团队成员应该向客户提供信息并帮助他们考虑质量的所有方面,包括性能和安全性等非功能性需求。最终决定取决于客户。团队可以通过采取简单的、循序渐进的方法来帮助客户做出正确的决定。敏捷测试手段进行尽可能简单的测试以验证某项功能是否存在或是否满足客户的质量标准(例如,性能)。
Some of us worked in software organizations where we, as testers and quality assurance staff, were asked to set quality standards. We believe this is backwards, because it’s up to the customer team to decide what level of quality they want to pay for. Testers and other team members should provide information to customers and help them consider all aspects of quality, including nonfunctional requirements such as performance and security. The ultimate decisions are up to the customer. The team can help the customer make good decisions by its taking a simple, step-by-step approach to its work. Agile testing means doing the simplest tests possible to verify that a piece of functionality exists or that the customer’s quality standard (e.g., performance) has been met.
第 9 章“支持团队的面向业务的测试工具包”和第 11 章“使用面向技术的测试评价产品”给出了测试工具的示例。
Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” and Chapter 11, “Critiquing the Product Using Technology-Facing Tests,” give examples of test tools.
简单并不意味着容易。对于测试人员来说,这意味着使用我们能找到的最轻量级的工具和技术来测试“刚好足够”,以完成这项工作。工具可以像电子表格或清单一样简单。我们需要自动化回归测试,但我们应该将它们降低到尽可能低的级别,以鼓励快速反馈。即使是简单的冒烟测试也可能足以实现面向业务的测试自动化。
Simple doesn’t mean easy. For testers, it means testing “just enough” with the lightest-weight tools and techniques we can find that will do the job. Tools can be as simple as a spreadsheet or a checklist. We need to automate regression tests, but we should push them down to the lowest level possible in order to encourage fast feedback. Even simple smoke tests might be enough for business-facing test automation.
探索性测试可用于了解您的应用程序并找出难以发现的错误,但要从基础知识、时间限制的旁路测试和评估边缘案例的进展情况开始。简单有助于我们将注意力集中在风险、投资回报和改进最痛苦的领域。
Exploratory testing can be used to learn about your application and ferret out hard-to-find bugs, but start with the basics, time-boxing side trips and evaluating how far to go with edge cases. Simplicity helps us keep our focus on risk, return on investment, and improving in the areas of greatest pain.
第 IV 部分,“测试自动化”,解释了如何构建“可行的”测试自动化策略。
Part IV, “Test Automation,” explains how to build a “doable” test automation strategy.
寻找方法来做得更好是敏捷测试人员心态的一部分。当然,整个团队都应该这样想,因为敏捷的中心核心是团队总是努力把工作做得更好。测试人员参与团队回顾,评估哪些运作良好,哪些需要添加或调整。测试人员将测试问题提交给整个团队来解决。团队通过使用过程改进实践(例如回顾和障碍积压)在测试和所有其他领域取得了最大的改进。一些改进想法可能会成为任务卡。对于更大的问题,团队一次只关注一两个问题,以确保他们解决了真正的问题,而不仅仅是表面现象。
Looking for ways to do a better job is part of an agile tester’s mind-set. Of course, the whole team should be thinking this way, because the central core of agile is that the team always tries to do better work. Testers participate in team retrospectives, evaluating what’s working well and what needs to be added or tweaked. Testers bring testing issues up for the whole team to address. Teams have achieved their greatest improvements in testing and all other areas through the use of process improvement practices such as retrospectives and impediment backlogs. Some improvement ideas might become task cards. For larger problems, teams focus on one or two issues at a time to make sure they solve the real problem and not just the symptom.
敏捷测试人员和他们的团队总是在寻找可以帮助他们增加更多价值或获得更好的客户投资回报的工具、技能或实践。敏捷开发的短迭代使得在几次迭代中尝试新事物变得更容易,并且看看它是否值得长期采用。
Agile testers and their teams are always on the lookout for tools, skills, or practices that might help them add more value or get a better return on the customer’s investment. The short iterations of agile development make it easier to try something new for a few iterations and see whether it’s worth adopting for the long term.
学习新技能和专业成长对敏捷测试人员来说很重要。他们利用许多可用的免费资源来提高他们的专业技能,例如探索性测试。他们参加各种会议、加入邮件列表、阅读文章、博客和书籍以获得新想法。他们寻找自动化(或从同事那里获得帮助以自动化)平凡或重复性任务的方法,以便他们有更多时间贡献他们宝贵的专业知识。
Learning new skills and growing professionally are important to agile testers. They take advantage of the many available free resources to improve their specialized skills, such as exploratory testing. They go to meetings and conferences, join mailing lists, and read articles, blogs, and books to get new ideas. They look for ways to automate (or get help from their coworkers to automate) mundane or repetitive tasks so they have more time to contribute their valuable expertise.
Pierre Veragren 是 iLevel by Weyerhaeuser 的 SQA 负责人,他发现了我们自己在敏捷团队中经常看到的一种品质:“AADD”,即敏捷注意力缺陷障碍。任何不能快速学习的东西都可能被认为是无用的。敏捷团队成员寻求投资回报,如果他们没有很快看到回报,他们就会继续前进。当您每两周甚至更频繁地交付生产就绪软件时,这并不是一个负面特征。
Pierre Veragren, an SQA Lead at iLevel by Weyerhaeuser, identified a quality we often see in agile teams ourselves: “AADD,” Agile Attention Deficit Disorder. Anything not learned quickly might be deemed useless. Agile team members look for return on investment, and if they don’t see it quickly, they move on. This isn’t a negative characteristic when you’re delivering production-ready software every two weeks or even more often.
回顾是一种关键的敏捷实践,它可以让团队利用昨天的经验在明天做得更好。敏捷测试人员利用这个机会提出与测试相关的问题,并要求团队集思广益解决这些问题。这是团队向自己提供反馈以持续改进的一种方式。
Retrospectives are a key agile practice that lets the team use yesterday’s experience to do a better job tomorrow. Agile testers use this opportunity to raise testing-related issues and ask the team to brainstorm ways to address them. This is a way for the team to provide feedback to itself for continual improvement.
我们将在第 19 章“总结迭代”中更多地讨论回顾以及它们如何帮助您的团队实践持续改进。
We’ll talk more about retrospectives and how they can help your team practice continuous improvement in Chapter 19, “Wrap Up the Iteration.”
当我们在瀑布环境中工作时,我们习惯于说,“对不起,我们现在不能做这个改变;要求被冻结。我们必须把它放在第一个补丁版本中。” 这让客户感到沮丧,因为他们意识到他们没有很好地预先定义所有需求。
When we worked in a waterfall environment, we got used to saying, “Sorry, we can’t make this change now; the requirements are frozen. We’ll have to put that in the first patch release.” It was frustrating for customers because they realized that they didn’t do a great job on defining all their requirements up front.
在为期两周的敏捷迭代中,我们可能不得不说,“好吧,为此写一张卡片,我们将在下一次迭代或下一次发布中完成,”但客户知道他们可以在需要时获得更改,因为他们控制优先级。
In a two-week agile iteration, we might have to say, “OK, write a card for that and we’ll do it in the next iteration or next release,” but customers know they can get their change when they want it because they control the priority.
响应变化是敏捷实践者的一个关键价值,但我们发现它是测试人员最难理解的概念之一。稳定性是测试人员所渴望的,这样他们就可以说,“我已经测试过了;我已经测试过了;” 完成。” 不断变化的需求是测试人员的噩梦。但是,作为敏捷测试人员,我们必须欢迎变化。周三,我们可能会开始故事 A 和 B然后下周五 C。到周五,客户可能已经重新确定优先级,现在想要故事 A、X 和 Y。只要我们继续与客户交谈,我们就可以处理这样的变化,因为我们与团队其他成员的工作节奏相同.
Responding to change is a key value for agile practitioners, but we’ve found that it’s one of the most difficult concepts for testers. Stability is what testers crave so that they can say, “I’ve tested that; it’s done.” Continuously changing requirements are a tester’s nightmare. However, as agile testers, we have to welcome change. On Wednesday, we might expect to start stories A and B and then C the next Friday. By Friday, the customer could have re-prioritized and now wants stories A, X, and Y. As long as we keep talking to the customer, we can handle changes like that because we are working at the same pace with the rest of team.
一些敏捷团队尝试在下一次迭代之前做好准备,可能是通过编写高级测试用例、捕获业务满意度条件或记录示例。这是一项棘手的工作,如果重新确定故事的优先级或进行重大更改,可能会导致时间浪费。然而,分布式团队尤其需要额外的反馈周期来为迭代做好准备。
Some agile teams try to prepare in advance of the next iteration, perhaps by writing high-level test cases, capturing business satisfaction conditions, or documenting examples. It’s a tricky business that might result in wasted time if stories are re-prioritized or greatly changed. However, distributed teams in particular need extra feedback cycles to get ready for the iteration.
有些团队有分析师,他们可以花更多时间与业务专家一起做一些预先计划。每个团队都必须在提前集思广益解决方案和在每次迭代的第一天从头开始之间取得平衡。敏捷测试人员顺应潮流并与团队合作以适应变化。
Some teams have analysts who can spend more time with the business experts to do some advance planning. Each team has to strike a balance between brainstorming solutions ahead of time and starting from scratch on the first day of each iteration. Agile testers go with the flow and work with the team to accommodate changes.
自动化测试是解决方案的关键之一。我们可以肯定地知道一件事:没有任何敏捷团队仅通过手动测试就能成功。我们需要强大的自动化,以便在使其有价值的时间范围内交付业务价值。
Automated testing is one key to the solution. One thing we know for sure: No agile team will succeed doing only manual testing. We need robust automation in order to deliver business value in a time frame that makes it valuable.
敏捷测试人员是自组织敏捷团队的一部分。团队文化灌输了敏捷测试理念。当程序员、系统管理员、分析师、数据库专家和客户团队不断思考测试和测试自动化时,测试人员就会享受到全新的视角。自动化测试很难,但当你拥有整个团队时,它会容易得多一起工作。当您有具有多种技能和多种观点的人来攻击它时,任何测试问题都更容易解决。
The agile tester is part of a self-organizing agile team. The team culture imbues the agile testing philosophy. When programmers, system administrators, analysts, database experts, and the customer team think continually about testing and test automation, testers enjoy a whole new perspective. Automating tests is hard, but it is much easier when you have the whole team working together. Any testing issue is easier to address when you have people with multiple skill sets and multiple perspectives attacking it.
当一个敏捷团队面临一个大问题时,也许是生产停滞或构建失败,这是每个人的问题。最高优先级的问题是整个团队要解决的问题。团队成员立即讨论问题并决定如何解决以及由谁来解决。
When an agile team faces a big problem, perhaps a production showstopper or a broken build, it’s everyone’s problem. The highest-priority issues are problems for the whole team to solve. Team members discuss the issue right away and decide how to and who will fix it.
毫无疑问,Lisa 的经理可以要求团队采用这种方法来解决其自动化问题,但团队自己可以提出最可行的计划。当团队创建自己的方法并致力于它时,其成员就会对测试采取新的态度。
There’s no doubt that Lisa’s manager could have mandated that the team take this approach to solving its automation problems, but the team itself can come up with the most workable plan. When the team creates its own approach and commits to it, its members adopt a new attitude toward testing.
当好人被允许做他们最好的工作时,项目就会成功。敏捷价值观和原则的创建旨在实现个人和团队的成功。敏捷团队成员应该感到安全,不必担心因错误而受到指责或失去工作。敏捷团队成员相互尊重并认可个人成就。敏捷团队中的每个人都应该有机会成长和发展他们的技能。敏捷团队以可持续的步伐工作,使他们能够遵循纪律严明的做法并保持全新的视角。正如敏捷宣言所述,我们重视个人和交互胜过流程和工具。
Projects succeed when good people are allowed to do their best work. Agile values and principles were created with the aim of enabling individual and team success. Agile team members should feel safe and not have to worry about being blamed for mistakes or losing their jobs. Agile team members respect each other and recognize individual accomplishments. Everyone on an agile team should have opportunities to grow and develop their skills. Agile teams work at a sustainable pace that lets them follow disciplined practices and keep a fresh perspective. As the Agile Manifesto states, we value individuals and interactions over processes and tools.
在软件开发的历史中,测试人员并不总是享有与开发团队中其他角色同等的地位。有些人将测试人员视为失败的程序员或软件开发领域的二等公民。懒得学习新技能和专业成长的测试人员会导致人们认为测试是一项低技能工作。甚至连“测试员”这个词也被避免了,优先使用“质量保证工程师”或“质量分析师”等职位以及“QA 部门”等团队名称。
In the history of software development, testers haven’t always enjoyed parity with other roles on the development team. Some people saw testers as failed programmers or second-class citizens in the world of software development. Testers who don’t bother to learn new skills and grow professionally contribute to the perception that testing is low-skilled work. Even the term “tester” has been avoided, with job titles such as “Quality Assurance Engineer” or “Quality Analyst” and team names such as “QA Department” given preference.
坚持真正敏捷哲学的敏捷团队给予所有团队成员同等的权重。敏捷测试人员知道他们为他们的团队贡献了独特的价值,而开发团队发现当他们的团队包括具有特定测试技能和背景的人时,他们会更加成功。例如,熟练的探索性测试人员可能会发现系统中无法通过自动功能测试检测到的问题。具有深厚测试经验的人可能会提出重要的问题,而这些问题对于没有测试经验的团队成员来说是不会想到的。测试知识是任何团队交付价值能力的组成部分之一。
Agile teams that adhere to the true agile philosophy give all team members equal weight. Agile testers know they contribute unique value to their teams, and development teams have found they are more successful when their team includes people with specific testing skills and background. For example, a skilled exploratory tester may discover issues in the system that couldn’t be detected by automated functional tests. Someone with deep testing experience might ask important questions that didn’t occur to team members without testing experience. Testing knowledge is one component of any team’s ability to deliver value.
在一个每个人都协作的团队中工作,你从头到尾参与项目,业务利益相关者与开发团队一起工作,整个团队负责质量和测试,在我们看来,这简直就是测试者的乌托邦。我们并不是唯一相信每个人都应该在工作中找到乐趣的人。敏捷开发奖励敏捷测试人员对其工作的热情。
Working on a team where everyone collaborates, where you are engaged in the project from start to finish, where business stakeholders work together with the development team, where the whole team takes responsibility for quality and testing, in our opinion, is nothing short of a tester’s Utopia. We’re not alone in believing that everyone should find joy in their work. Agile development rewards the agile tester’s passion for her work.
我们作为敏捷测试人员的工作特别令人满意,因为我们的观点和技能让我们能够为我们的团队增加真正的价值。在下一节中,我们将探讨如何。
Our jobs as agile testers are particularly satisfying because our viewpoint and skills let us add real value to our teams. In the next section, we’ll explore how.
这些原则给团队带来了什么?它们共同带来了商业价值。在敏捷开发中,整个团队负责交付高质量的软件,让客户满意并使业务更有利可图。这反过来又为企业带来了新的优势。
What do these principles bring to the team? Together, they bring business value. In agile development, the whole team takes responsibility for delivering high-quality software that delights customers and makes the business more profitable. This, in turn, brings new advantages for the business.
团队成员身兼数职,敏捷开发倾向于避免按专业对人进行分类。即使是短迭代和频繁发布,也很容易在客户团队的期望与团队交付的内容之间形成差距。使用测试来推动开发有助于防止这种情况发生,但您仍然需要正确的测试。
Team members wear many hats, and agile development tends to avoid classifying people by specialty. Even with short iterations and frequent releases, it’s easy to develop a gap between what the customer team expects and what the team delivers. Using tests to drive development helps to prevent this, but you still need the right tests.
敏捷测试人员不仅从将接受解决方案的利益相关者的角度考虑系统,而且还掌握了开发团队面临的技术约束和实施细节。程序员专注于让事情运转起来。如果他们按照正确的要求进行编码,客户会很高兴。不幸的是,客户通常不善于表达他们的需求。用错误的测试推动开发不会产生预期的结果。敏捷测试人员尽早并经常向客户和开发人员提出问题,并帮助将答案塑造成正确的测试。
Agile testers not only think about the system from the viewpoint of stakeholders who will live with the solution but they also have a grasp of technical constraints and implementation details that face the development team. Programmers focus on making things work. If they’re coding to the right requirements, customers will be happy. Unfortunately, customers aren’t generally good at articulating their requirements. Driving development with the wrong tests won’t deliver the desired outcome. Agile testers ask questions of both customers and developers early and often, and help shape the answers into the right tests.
与传统瀑布项目的测试人员相比,敏捷测试人员采用更加集成、面向团队的方法。他们使自己的技能和经验适应团队和项目。将程序员视为对手,或者坐等工作交给她,或者期望花更多时间计划而不是做的测试人员可能会坚持她在传统项目中学到的技能,并且不会在敏捷团队中待太久.
Agile testers take a much more integrated, team-oriented approach than testers on traditional waterfall projects. They adapt their skills and experience to the team and project. A tester who views programmers as adversaries, or sits and waits for work to come to her, or expects to spend more time planning than doing, is likely to cling to skills she learned on traditional projects and won’t last long on an agile team.
在故事估算和计划会议期间,敏捷测试人员从多个角度审视每个特性:业务、最终用户、生产支持和程序员。他们考虑企业面临的问题以及软件如何解决这些问题。他们提出的问题会推翻客户和开发团队所做的假设。在每次迭代开始时,他们帮助确保客户提供明确的需求和示例,并帮助开发团队将这些转化为测试。测试推动开发,测试结果提供团队进度的反馈。测试人员帮助提出问题,以便不会忽视任何测试;它不仅仅是功能测试。客户并不总是知道他们应该提及他们的性能和可靠性需求或安全问题,但测试人员会考虑询问这些。测试人员还使测试方法和工具尽可能简单和轻便。在迭代结束时,测试人员验证是否已完成最低限度的测试。
During story estimating and planning sessions, agile testers look at each feature from multiple perspectives: business, end user, production support, and programmer. They consider the problems faced by the business and how the software might address them. They raise questions that flush out assumptions made by the customer and developer teams. At the start of each iteration, they help to make sure the customer provides clear requirements and examples, and they help the development team turn those into tests. The tests drive development, and test results provide feedback on the team’s progress. Testers help to raise issues so that no testing is overlooked; it’s more than functional testing. Customers don’t always know that they should mention their performance and reliability needs or security concerns, but testers think to ask about those. Testers also keep the testing approach and tools as simple and lightweight as possible. By the end of the iteration, testers verify that the minimum testing was completed.
敏捷团队中角色之间的界限是模糊的。其他团队成员可能擅长测试人员执行的相同活动。例如,分析师和程序员也编写面向业务的测试。只要执行了所有测试活动,敏捷团队就不一定需要主要将自己标识为测试人员的成员。然而,我们发现团队受益于专业测试人员开发的技能。我们讨论的敏捷原则和价值观将帮助任何团队做好测试和交付价值的工作。
Lines between roles on an agile team are blurred. Other team members might be skilled at the same activities that testers perform. For example, analysts and programmers also write business-facing tests. As long as all testing activities are performed, an agile team doesn’t necessarily require members who identify themselves primarily as testers. However, we have found that teams benefit from the skills that professional testers have developed. The agile principles and values we’ve discussed will help any team do a good job of testing and delivering value.
在本章中,我们介绍了敏捷测试人员的原则以及我们认为敏捷测试人员需要具备的价值观,以便有效地为敏捷团队做出贡献。
In this chapter, we covered principles for agile testers and the values we think an agile tester needs to possess in order to contribute effectively to an agile team.
“敏捷测试思维模式”是以客户为中心、以结果为导向、工匠般的、协作的、创造性的、渴望学习的,并且热衷于及时交付业务价值。
An “agile testing mind-set” is customer-focused, results-oriented, craftsman-like, collaborative, creative, eager to learn, and passionate about delivering business value in a timely manner.
态度很重要,它模糊了测试人员、程序员和敏捷团队中其他角色之间的界限。
Attitude is important, and it blurs the lines between testers, programmers, and other roles on an agile team.
敏捷测试人员应用敏捷的价值观和原则,如反馈、沟通、勇气、简单、享受和交付价值,以帮助团队识别和交付每个故事的客户需求。
Agile testers apply agile values and principles such as feedback, communication, courage, simplicity, enjoyment, and delivering value in order to help the team identify and deliver the customer requirements for each story.
敏捷测试人员以其独特的观点和面向团队的方法为他们的团队和组织增加价值。
Agile testers add value to their teams and their organizations with their unique viewpoint and team-oriented approach.
当软件开发组织实施敏捷开发时,测试或 QA 团队通常需要最长的时间才能完成过渡。独立的 QA 团队已经在许多组织中根深蒂固。当他们开始适应新的敏捷组织时,他们会遇到难以接受的文化差异。在第 II 部分中,我们讨论了引入变更以及您在向敏捷过渡时可能遇到的一些障碍。培训是进行转型的组织的重要组成部分,但往往被遗忘。也很难看出审计和流程改进框架等现有流程将如何在敏捷环境中发挥作用。从一个独立的 QA 团队到一个集成的敏捷团队是一个巨大的变化。
When software development organizations implement agile development, the testing or QA team often takes the longest to make the transition. Independent QA teams have become entrenched in many organizations. When they start to adapt to a new agile organization, they encounter cultural differences that are difficult for them to accept. In Part II, we talk about introducing change and some of the barriers you might encounter when transitioning to agile. Training is a big part of what organizations making the transition need, and it’s often forgotten. It’s also hard to see how existing processes such as audits and process improvement frameworks will work in the agile environment. Going from an independent QA team to an integrated agile team is a huge change.
第 4 章“团队后勤”讨论了团队结构,例如测试人员在团队中的实际位置,以及关于测试人员与开发人员比率的永无止境的问题。我们还将讨论招聘测试人员以及在成功的敏捷测试人员中寻找什么。
Chapter 4, “Team Logistics,” talks about the team structure, such as where a tester actually fits into the team, and the never-ending question about tester-developer ratio. We’ll also talk about hiring testers and what to look for in a successful agile tester.
传统的测试活动,例如记录错误、跟踪指标和编写测试计划,可能看起来不太适合敏捷项目。我们介绍了一些可能需要特别注意和注意的典型流程,并讨论了如何调整现有的质量流程。
Traditional testing activities, such as logging bugs, keeping track of metrics, and writing test plans, might not seem like a good fit in an agile project. We introduce some of the typical processes that might need special care and attention and discuss how to adapt existing quality processes.
您可以期望找到一些方法,让习惯于传统瀑布式开发环境的测试人员和测试团队可以改变他们的组织结构和文化,从而从敏捷开发中受益并为敏捷开发增加价值。
You can expect to find ways that testers and test teams accustomed to a traditional waterfall type of development environment can change their organizational structure and culture to benefit from and add value to agile development.
许多组织影响都会影响项目,无论它使用敏捷还是传统的分阶段或门控方法。组织和团队文化会阻碍向敏捷方法的平稳过渡。在本章中,我们将讨论可以直接影响测试人员在敏捷团队中的角色的因素。
Many organizational influences can impact a project, whether it uses an agile or a traditional phased or gated approach. Organizational and team culture can block a smooth transition to an agile approach. In this chapter, we discuss factors that can directly affect a tester’s role on an agile team.
组织文化由其价值观、规范和假设来定义。一个组织的文化支配着人们如何沟通、相互关联和做出决策,通过观察员工行为很容易看出这一点。
An organizational culture is defined by its values, norms, and assumptions. An organization’s culture governs how people communicate, interrelate, and make decisions, and it is easily seen by observing employee behavior.
组织的文化可以影响敏捷团队的成功。敏捷团队最适合允许独立思考的组织。例如,如果一家公司采用层级结构并鼓励对其所有项目采用指令式管理方式,那么敏捷团队可能会举步维艰。组织过去的经验也会影响新敏捷团队的成功。如果一家公司尝试敏捷但结果不佳,人们会怀疑是否会再次尝试,并举出失败的例子。他们甚至可能会积极反对它。
The culture of an organization can impact the success of an agile team. Agile teams are best suited for organizations that allow independent thinking. For example, if a company has a hierarchical structure and encourages a directive management style for all its projects, agile teams will probably struggle. Past experiences of the organization will also affect the success of a new agile team. If a company tried agile and had poor results, people will be suspicious of trying it again, citing examples of why it didn’t work. They might even actively campaign against it.
在尝试实施敏捷过程时,组织文化经常被忽视,这让人们想知道为什么它没有像承诺的那样工作。很难改变既定的流程,尤其是当个人觉得他们对现状有利害关系时。每个职能部门都发展出满足其需求的亚文化和流程。他们对自己的工作方式感到满意。恐惧是一种强烈的情绪,如果不加以解决,它可能会危及向敏捷的过渡。如果团队成员觉得新的敏捷过程威胁到他们的工作,他们就会抵制这种变化。
Organizational culture is too frequently not considered when attempts are made to implement an agile process, leaving people wondering why it didn’t work as promised. It’s hard to change established processes, especially if individuals feel they have a stake in the status quo. Each functional group develops a subculture and processes that meet their needs. They’re comfortable with the way they work. Fear is a powerful emotion, and if it is not addressed, it can jeopardize the transition to agile. If team members feel that a new agile process threatens their jobs, they’ll resist the change.
我们将具体讨论组织文化如何影响在敏捷环境中工作的测试人员。参考书目包含处理可能影响团队的其他文化方面的资源。
We’ll talk specifically about how organizational culture affects testers working in an agile environment. The bibliography contains resources that deal with other cultural aspects that may affect teams.
从组织如何确定可接受的软件质量水平的角度考虑组织的质量理念。它能容忍质量差吗?它是否考虑了客户的质量要求,还是只关心尽快将产品送到客户手中?
Consider an organization’s quality philosophy in terms of how it determines the acceptable level of software quality. Does it tolerate poor quality? Does it take customers’ quality requirements into account, or is it just concerned with getting the product into the customers’ hands as fast as it can?
当一个组织缺乏整体质量理念并迫使团队在不考虑质量的情况下推出产品时,测试人员会感到压力。试图在这样的环境中使用敏捷开发的团队面临着一场艰苦的战斗。
When an organization lacks an overall quality philosophy and pressures teams to get the product out without regard to quality, testers feel the pinch. A team that tries to use agile development in such an environment faces an uphill battle.
一些组织拥有强大的、独立的测试团队,他们拥有很大的权力。这些团队及其经理可能认为敏捷开发会夺走这种力量。他们可能担心敏捷与他们的质量理念背道而驰。评估您组织的质量理念和执行它的团队的理念。
Some organizations have strong, independent test teams that wield a lot of power. These teams, and their managers, might perceive that agile development will take that power away. They might fear that agile runs contrary to their quality philosophy. Evaluate your organization’s quality philosophy and the philosophy of the teams that enforce it.
每个人都重视质量的公司将更容易过渡到敏捷。如果任何一个团队承担了质量责任,他们就必须学会与团队中的其他人分享这一点才能取得成功。
Companies in which everyone values quality will have an easier time transitioning to agile. If any one group has assumed ownership of quality, they’ll have to learn to share that with everyone else on the team in order to succeed.
在第 1 章中,“到底什么是敏捷测试?”,我们谈到了整个团队的质量方法。对于许多测试人员和 QA 团队来说,这意味着思想从拥有质量转变为在定义和维护质量方面发挥参与作用。对于许多测试人员和 QA 团队来说,如此剧烈的态度转变是困难的。
In Chapter 1, “What Is Agile Testing, Anyway?,” we talked about the whole-team approach to quality. For many testers and QA teams, this means a mind shift from owning quality to having a participatory role in defining and maintaining quality. Such a drastic shift in attitude is difficult for many testers and QA teams.
一直在传统环境中工作的测试人员可能很难适应他们的新角色和活动。如果他们来自开发和 QA 存在敌对关系的组织,则可能很难从事后才想到(如果有的话)转变为团队不可或缺的一部分。程序员和测试人员很难学会相互信任。
Testers who have been working in a traditional setting might have a hard time adjusting to their new roles and activities. If they’ve come from an organization where development and QA have an adversarial relationship, it may be difficult to change from being an afterthought (if thought of at all) to being an integral part of the team. It can be difficult for both programmers and testers to learn to trust each other.
关于无法适应敏捷实践的程序员已经有很多观察——但是对于习惯于根据需求文档构建测试脚本的测试人员呢?他们能学会问问题作为代码吗正在建设中?不改变测试方法的测试人员很难与开发团队的其他成员紧密合作。
Much has been observed about programmers who can’t adapt to agile practices—but what about testers who are used to building test scripts according to a requirements document? Can they learn to ask the questions as the code is being built? Testers who don’t change their approach to testing have a hard time working closely with the rest of the development team.
习惯于通过用户界面仅进行手动测试的测试人员可能不了解敏捷开发固有的自动化方法。这些测试人员需要很大的勇气来面对他们不断变化的角色,因为变化意味着在他们的舒适区之外发展新的技能组合。
Testers who are used to doing only manual testing through the user interface might not understand the automated approach that is intrinsic to agile development. These testers need a lot of courage in order to face their changing roles, because changing means developing new skill sets outside their comfort zones.
尽管有许多文化问题需要考虑,但大多数 QA 团队都将重点放在流程改进上,敏捷项目通过使用回顾等工具鼓励持续改进和适应性。大多数质量保证专业人员都渴望利用他们所学到的知识并加以改进。这些人具有足够的适应性,不仅可以生存,而且可以在敏捷项目中茁壮成长。
Although there are many cultural issues to consider, most QA teams have a focus on process improvement, and agile projects encourage continuous improvements and adaptability through the use of tools like retrospectives. Most quality assurance professionals are eager to take what they’ve learned and make it better. These people are adaptable enough to not only survive, but to thrive in an agile project.
如果您的组织专注于学习,它将鼓励持续的流程改进。它可能会比那些更重视如何应对危机而不是改进流程的组织更快地采用敏捷。
If your organization focuses on learning, it will encourage continual process improvement. It will likely adopt agile much more quickly than organizations that put more value on how they react to crises than on improving their processes.
如果您是一个没有有效质量理念的组织中的测试人员,您可能很难让质量实践被接受。敏捷方法将为您提供一种机制,用于引入良好的面向质量的实践。
If you are a tester in an organization that has no effective quality philosophy, you probably struggle to get quality practices accepted. The agile approach will provide you with a mechanism for introducing good quality-oriented practices.
测试人员需要时间和培训,就像其他学习敏捷项目的人一样。如果您管理的团队包括测试人员,一定要给他们足够的支持。测试人员通常不会在新项目开始时被引入,然后被期望能够适应已经一起工作了几个月的团队。为了帮助测试人员进行调整,您可能需要引入经验丰富的敏捷测试教练。雇用以前在敏捷团队工作过并且可以担任导师和老师的人将帮助测试人员融入新的敏捷文化,无论他们是与现有团队一起过渡到敏捷,还是加入新的敏捷开发团队。
Testers need time and training, just like everyone else who is learning to work on an agile project. If you’re managing a team that includes testers, be sure to give them plenty of support. Testers are often not brought in at the beginning of a greenfield project and are then expected to just fit into a team that has been working together for months. To help testers adjust, you may need to bring in an experienced agile testing coach. Hiring someone who has previously worked on an agile team and can serve as a mentor and teacher will help testers integrate with the new agile culture, whether they’re transitioning to agile along with an existing team or joining a new agile development team.
传统的测试团队习惯于在项目结束时进行快速而激烈的测试,这转化为周末和晚上的工作。在这个项目结束测试阶段,一些组织定期要求他们的团队每周投入 50、60 或更多小时来努力赶上最后期限。组织通常将加班时间视为衡量个人承诺的标准。这与敏捷价值观相冲突,敏捷价值观围绕着让人们始终把工作做到最好。
Traditional test teams are accustomed to fast and furious testing at the end of a project, which translates into working weekends and evenings. During this end-of-project testing phase, some organizations regularly ask their teams to put in 50, 60, or more hours each week to try to meet a deadline. Organizations often look at overtime as a measure of an individual’s commitment. This conflicts with agile values that revolve around enabling people to do their best work all the time.
在敏捷项目中,我们鼓励您以可持续的速度工作。这意味着团队以一致的速度工作,保持恒定的速度,从而保持高质量标准。新的敏捷团队往往对他们可以完成的事情过于乐观,并且报名参加了太多的工作。经过一两次迭代后,他们学会了注册足够的工作,因此不需要加班来完成他们的任务。每周工作 40 小时是 XP 团队正常的可持续发展速度;如果一周又一周地付出努力,人们就可以在长期内完成最多的工作,同时交付良好的价值。
In agile projects, you are encouraged to work at a sustainable pace. This means that teams work at a consistent pace that sustains a constant velocity that permits maintaining a high-quality standard. New agile teams tend be overly optimistic about what they can accomplish and sign up for too much work. After an iteration or two, they learn to sign up for just enough work so no overtime is needed to complete their tasks. A 40-hour week is the normal sustainable pace for XP teams; it is the amount of effort that, if put in week in and week out, allows people to accomplish the most work over the long haul while delivering good value.
团队可能需要不时地为不可持续的节奏而努力,但这应该是例外,而不是常态。如果需要短期加班,整个团队应该加班。如果是冲刺的最后一天并且有些故事没有测试,那么整个团队都应该熬夜完成测试,而不仅仅是测试人员。使用本书中推荐的实践和技术来学习如何规划测试和开发,并让测试“跟上”编码。在您的团队更好地管理其工作量和速度之前,预算额外的时间以帮助平衡步伐。
Teams might need to work for short bursts of unsustainable pace now and then, but it should be the exception, not the norm. If overtime is required for short periods, the whole team should be working extra hours. If it’s the last day of the sprint and some stories aren’t tested, the whole team should stay late to finish the testing, not just the testers. Use the practices and techniques recommended throughout this book to learn how to plan testing along with development and allow testing to “keep up” with coding. Until your team gets better at managing its workload and velocity, budget in extra time to help even out the pace.
在传统的软件开发中,开发团队与客户之间的关系更像是供应商关系。即使客户是内部客户,它也更像是两个独立的公司,而不是两个致力于创造商业价值的共同目标的团队。
In traditional software development, the relationship between the development teams and their customers is more like a vendor-supplier relationship. Even if the customer is internal, it can feel more like two separate companies than two teams working on a common goal of producing business value.
敏捷开发依赖于客户或至少是他们的代理人的密切参与。敏捷团队邀请客户进行协作,如果可能的话在同一地点工作,并密切参与开发过程。双方都了解彼此的长处和短处。
Agile development depends on close involvement from customers or, at the very least, their proxies. Agile teams have invited customers to collaborate, work in the same locations if possible, and be intimately involved with the development process. Both sides learn each other’s strengths and weaknesses.
这种关系的变化需要双方都认可,不管客户是内部客户还是外部客户。开放的关系对于敏捷项目的成功至关重要,在敏捷项目中,客户团队和开发团队之间的关系更像是一种伙伴关系,而不是供应商与供应商之间的关系。
This change in the relationships needs to be recognized by both sides, and it doesn’t matter whether the customer is internal or external. An open relationship is critical to the success of an agile project, where the relationship between the customer team and the development team is more like a partnership than a vendor-supplier relationship.
拥有一些有代表性的领域专家,同时让所有利益相关者不断了解情况,是开发人员与客户成功协作的一种方法。我们将在第五部分讨论其他内容。客户对于敏捷项目的成功至关重要。他们优先考虑将要构建的内容,并对产品质量拥有最终决定权。测试人员与客户密切合作,了解需求并定义验收测试,以证明满足条件。测试活动是开发团队与客户团队关系的关键。这就是测试专业知识对敏捷团队如此重要的原因。
Having a few representative domain experts, while keeping all stakeholders continually informed, is one approach to successful developer-customer collaboration. We’ll talk about others in Part V. Customers are critical to the success of your agile project. They prioritize what will be built and have the final say in the quality of the product. Testers work closely with customers to learn requirements and define acceptance tests that will prove that conditions of satisfaction are met. Testing activities are key to the development team-customer team relationship. That’s why testing expertise is so essential to agile teams.
组织的规模对项目的运行方式和公司结构的成熟度有很大的影响。组织越大,结构越趋于分层。随着自上而下的沟通渠道的发展,报告结构变得具有指导性,并且与技术和业务之间的协作不太兼容。
The size of an organization can have great impact on how projects are run and how the structure of a company matures. The larger the organization, the more hierarchical the structure tends to be. As top-down communication channels are developed, the reporting structures become directive and less compatible with collaboration between technology and business.
一些敏捷流程提供了促进团队间沟通的方法。例如,Scrum 有“Scrum of Scrums”,来自多个团队的代表每天进行协调。
Some agile processes provide ways to facilitate inter-team communication. For example, Scrum has the “Scrum of Scrums,” where representatives from multiple teams coordinate on a daily basis.
如果您在一个大型组织中工作,其中测试团队或其他专业资源与编程团队是分开的,请努力寻找保持联系的方法。例如,如果您的数据库团队是完全独立的,您需要找到一种与数据库专家密切合作的方式,以便及时获得您需要的东西。
If you work in a large organization where the test teams or other specialized resources are separate from the programming teams, work to find ways to keep in constant touch. For example, if your database team is completely separate, you need to find a way to work closely with the database specialists in order to get what you need in a timely manner.
另一个在大公司中更常见的问题是,客户可能不像在小公司那样容易接近。当您尝试收集需求和示例并寻求让客户参与整个开发周期时,这是一个很大的障碍。一种解决方案是让具有领域专业知识的测试人员或分析师充当客户代理。通信工具也可以帮助处理此类情况。寻找创造性的方法来克服大公司固有的问题。
Another problem that tends to be more common in large companies is that customers might not be as accessible as they are in smaller companies. This is a big obstacle when you try to gather requirements and examples and seek to get customer involvement throughout the development cycle. One solution is to have testers or analysts with domain expertise act as customer proxies. Communication tools can help deal with such situations as well. Look for creative ways to overcome the problems inherent in big companies.
第 16 章,“开始行动”,描述了一个大型组织如何使用功能分析师来缓解由于远程客户引起的问题。
Chapter 16, “Hit the Ground Running,” describes how one large organization uses functional analysts to mitigate problems due to remote customers.
对于大型软件开发商店,敏捷开发通常首先在一个团队或几个团队中实施。如果您的敏捷团队必须使用其他方法(例如分阶段或门控开发)与其他团队进行协调,那么您将面临一系列额外的挑战。如果一些外部团队往往功能失调,那就更难了。即使整个公司都采用了敏捷,一些团队也比其他团队更成功地完成了转型。
With large software development shops, agile development is often first implemented in one team or just a few teams. If your agile team has to coordinate with other teams using other approaches such as phased or gated development, you have an extra set of challenges. If some of the external teams tend to be dysfunctional, it’s even harder. Even when an entire company adopts agile, some teams make the transition more successfully than others.
您的团队也可能会遇到专家团队的抵制,这些专家团队对他们的特定孤岛感到保护。Lisa 与一个团队交谈,该团队的成员无法从他们公司的配置管理团队获得任何帮助,这显然是一个主要障碍。一些开发团队被禁止直接与客户交谈。
Your team might also run into resistance from specialist teams that are feeling protective of their particular silos. Lisa talked to a team whose members could not get any help from their company’s configuration management team, which was obviously a major obstacle. Some development teams are barred from talking directly to customers.
如果第三方在您的团队正在使用的同一系统上工作,他们的文化也会导致冲突。也许您的团队是第三方,而您正在为客户开发软件。您将需要考虑如何减轻基于文化的差异。第 V 部分更详细地介绍了与其他团队和第三方合作,但这里有一些想法可以帮助您入门。
If third parties are working on the same system your team is working on, their cultures can also cause conflicts. Perhaps your team is the third party, and you’re developing software for a client. You will need to think about how to mitigate culture-based differences. Part V goes into more detail about working with other teams and third parties, but here are a few ideas to get you started.
高级计划如果您必须与其他团队协调,则需要在发布计划期间或迭代开始前花时间与他们合作。您需要时间来调整您自己的流程以与其他人的流程一起工作,并且他们可能需要更改他们的流程以适应您的要求。考虑安排对共享资源(如性能测试专家或负载测试环境)的访问,并围绕其他人的日程安排自己的工作。您的利益相关者可能期望您自己的敏捷过程不包括的某些可交付成果,例如正式的测试计划。一些额外的计划将帮助您克服这些文化差异。
Advanced Planning If you have to coordinate with other teams, you will need to spend time during release planning, or before the start of an iteration, to work with them. You need time to adapt your own processes to work with others’ processes, and they might need to change their processes to accommodate your requests. Consider arranging access to shared resources such as performance test specialists or load test environments, and plan your own work around others’ schedules. Your stakeholders might expect certain deliverables, such as formal test plans, that your own agile process doesn’t include. Some extra planning will help you to work through these cultural differences.
第 15 章,“发布或主题规划中的测试人员活动”和第 16 章,“开始行动”,讨论测试人员可以做什么来帮助规划和与其他团队协调。
Chapter 15, “Tester Activities in Release or Theme Planning,” and Chapter 16, “Hit the Ground Running,” talk about what testers can do to help with planning and coordinating with other teams.
现在行动,稍后道歉我们对提出可能引起麻烦的建议犹豫不决,但通常在大型组织中,官僚主义的轮子如此转动慢慢地,您的团队可能不得不找出并实施自己的解决方案。例如,无法与配置管理团队合作的团队只是实施了自己的内部构建流程,并继续努力将其与官方认可的流程相集成。
Act Now, Apologize Later We hesitate to make suggestions that might cause trouble, but often in a large organization, the bureaucratic wheels turn so slowly that your team might have to figure out and implement its own solutions. For example, the team that couldn’t get cooperation from the configuration management team simply implemented its own internal build process and kept working on getting it integrated with the officially sanctioned process.
如果没有官方渠道可以获得您需要的东西,那么是时候发挥创意了。也许测试人员以前从未直接与客户交谈过。尝试自己安排一次会面,或者找一个可以充当客户代理人或中间人的人。
If there aren’t official channels to get what you need, it’s time to get creative. Maybe testers have never talked directly to customers before. Try to arrange a meeting yourself, or find someone who can act as a customer proxy or go-between.
在敏捷项目中,重要的是每个开发团队都感到有权做出决策。如果您是一名经理,并且希望您的敏捷团队取得成功,请让他们自由地采取行动并做出创造性的反应。组织文化必须适应这种变化,敏捷项目才能成功。
In an agile project, it is important for each development team to feel empowered to make decisions. If you’re a manager and you want your agile teams to succeed, set them free to act and react creatively. The culture of an organization must adapt to this change for an agile project to be successful.
第 4 章,“团队后勤”,更多地讨论了独立的职能团队以及它们如何影响敏捷测试人员。
Chapter 4, “Team Logistics,” talks more about separate functional teams and how they affect the agile tester.
任何变革都面临成功的障碍。正如我们在上一节中讨论的那样,组织文化可能是需要克服的最大障碍。组织文化一旦确立,就很难改变。它的形成需要时间,一旦到位,员工就会致力于这种文化,这使得它极难改变。
Any change faces barriers to success. Organizational culture, as we discussed in the previous section, might be the largest obstacle to overcome. Once organizational culture has become well established, it’s very hard to change. It took time for it to form, and once in place, employees become committed to the culture, which makes it extremely resistant to alteration.
本节讨论您的测试人员和 QA 团队可能遇到的采用敏捷开发方法的具体障碍。
This section discusses specific barriers to adoption of agile development methods that can be encountered by your testers and QA teams.
出于多种原因,测试人员坚持独立 QA 团队的概念,但主要原因是恐惧,具体来说:
Testers cling to the concept of an independent QA team for many reasons, but the main reason is fear, specifically:
担心他们会失去 QA 身份
Fear that they will lose their QA identity
担心如果他们向开发经理报告,他们将失去支持而程序员将获得优先权
Fear that if they report to a development manager, they will lose support and programmers will get priority
担心他们缺乏在敏捷团队中工作的技能并会丢掉工作
Fear that they lack the skills to work in an agile team and will lose their jobs
担心当他们分散到开发团队时,他们得不到所需的支持
Fear that when they’re dispersed into development teams they won’t get the support they need
担心他们和他们的经理会在新组织中迷失方向
Fear that they, and their managers, will get lost in the new organization
我们经常听到 QA 经理提出诸如“我的公司正在实施敏捷开发”之类的问题。我的角色如何适应?” 这与“失去身份”的恐惧直接相关。
We often hear of QA managers asking questions such as, “My company is implementing agile development. How does my role fit in?” This is directly related to the “loss of identity” fears.
Chapter 4, “Team Logistics,” covers ideas that can be used to help people adapt.
我们从经验中知道,新团队往往缺少可能对其成功至关重要的专家或专业知识。Lisa 的团队遇到了如此大的障碍,唯一能做的就是坐下来问:“我们的团队中缺少什么角色阻碍了我们?我们需要什么?另一个开发人员、另一个测试人员、数据库设计人员?” 我们都知道测试是一个广阔的领域。也许您需要在敏捷团队中有测试经验的人。或者您可能需要一名性能测试专家。花时间分析你的产品需要哪些角色才能成功,这一点很重要,如果你需要从团队外部填补这些角色,那就去做吧。
We know from experience that new teams are often missing specialists or expertise that might be key to their success. Lisa’s team has run into obstacles so large that the only thing to do was sit back and ask, “What role are we missing on our team that is holding us back? What do we need? Another developer, another tester, a database designer?” We all know that testing is a vast field. Maybe you need someone experienced in testing on an agile team. Or maybe you need a performance testing specialist. It’s critical that you take the time to analyze what roles your product needs to be successful, and if you need to fill them from outside the team, do it.
至关重要的是,产品团队中的每个人都了解自己的角色,或者弄清楚自己的角色是什么,因为他们现在是新敏捷团队的一员。这样做需要时间和培训。
It’s critical that everyone already on the product team understand their role or figure out what their role is now that they’re part of a new agile team. Doing this requires time and training.
我们在Agile 2007 的“会议中的会议”中主持了一次会议,询问人们在他们的敏捷团队中遇到了哪些与测试相关的问题。一位与会者告诉我们,他们按照敏捷文献所提倡的那样拆分了测试组织。但是,他们在没有任何培训的情况下将测试人员放入开发单元;三个月内,所有测试人员都辞职了,因为他们不了解自己的新角色。通过正确的培训和辅导,可以避免此类问题。
We hosted a session in the “Conference within a Conference” at Agile 2007 that asked people what testing-related problems they were having on their agile teams. One of the attendees told us that they split up their test organization as advocated by the agile literature. However, they put the testers into development units without any training; within three months, all of the testers had quit because they didn’t understand their new roles. Problems like these can be prevented with the right training and coaching.
当我们开始与我们的第一个敏捷团队合作时,没有太多资源可以帮助我们了解敏捷测试人员应该做什么或者我们应该如何与我们的团队一起工作。今天,你可以找到很多实践者,他们可以帮助训练测试人员适应敏捷环境,帮助测试团队进行敏捷转型。本地用户组、会议、研讨会、在线指导和邮件列表都为想要学习的测试人员和管理人员提供了宝贵的资源。在需要时不要害怕寻求帮助。良好的指导会给您带来良好的投资回报。
When we started working with our first agile teams, there weren’t many resources available to help us learn what agile testers should do or how we should work together with our teams. Today, you can find many practitioners who can help train testers to adapt to an agile environment and help test teams make the agile transition. Local user groups, conferences, seminars, online instruction, and mailing lists all provide valuable resources to testers and managers wanting to learn. Don’t be afraid to seek help when you need it. Good coaching gives a good return on your investment.
并非所有的敏捷团队都是一样的。敏捷开发有很多不同的方法,例如 XP、Scrum、Crystal、FDD、DSDM、OpenUP 和各种那些的混合。在我们看来,一些自称为“敏捷”的团队并没有真正实践敏捷。许多团队只是采用对他们有用的做法,而不管原始来源如何,或者他们发明了自己的做法。这很好,但如果他们不遵循任何敏捷核心价值观和原则,我们就会质疑给他们贴上敏捷标签。每月发布和免除文档并不等同于敏捷开发!
Not all agile teams are the same. There are lots of different approaches to agile development, such as XP, Scrum, Crystal, FDD, DSDM, OpenUP, and various mixes of those. Some self-titled “agile” teams are not, in our opinion, really practicing agile. Plenty of teams simply adopt practices that work for them regardless of the original source, or they invent their own. That’s fine, but if they don’t follow any of the core agile values and principles, we question giving them an agile label. Releasing every month and dispensing with documentation does not equate to agile development!
如果不同的团队成员对什么是“敏捷”、他们应该使用哪些实践或应该如何实践这些实践有相反的看法,就会有麻烦。例如,如果你是一名测试人员,正在推动团队实施持续集成,但程序员根本拒绝尝试,那么你就处于不利地位。如果您是一名未能成功参与某些实践的程序员,例如通过面向业务的测试来推动开发,那么您也会陷入冲突。
If different team members have opposing notions of what constitutes “agile,” which practices they should use, or how those practices are supposed to be practiced, there’s going to be trouble. For example, if you’re a tester who is pushing for the team to implement continuous integration, but the programmers simply refuse to try, you’re in a bad spot. If you’re a programmer who is unsuccessful at getting involved in some practices, such as driving development with business-facing tests, you’re also in for conflict.
团队必须就如何继续进行达成共识,以便成功过渡到敏捷。许多敏捷开发实践是协同的,因此如果单独使用它们,它们可能无法提供团队正在寻找的好处。也许团队可以同意对给定迭代次数的某些实践进行试验并评估结果。它可以决定寻求外部输入,以帮助他们了解这些做法以及它们如何组合在一起。不同的观点对团队有好处,但每个人都需要朝着同一个方向前进。
The team must reach consensus on how to proceed in order to make a successful transition to agile. Many of the agile development practices are synergistic, so if they are used in isolation, they might not provide the benefits that teams are looking for. Perhaps the team can agree to experiment with certain practices for a given number of iterations and evaluate the results. It could decide to seek external input to help them understand the practices and how they fit together. Diverse viewpoints are good for a team, but everyone needs to be headed in the same direction.
我们采访过的一些人描述了传统软件开发组织实施敏捷开发过程时经常出现的“迷你瀑布”现象。该组织用两周或四个星期的开发周期取代了六个月或一年的开发周期,并试图将所有传统的 SDLC 阶段压缩到这么短的时间内。自然地,他们继续遇到与以前相同的问题。图 3-1显示了迷你瀑布的“理想”版本,其中有一个代码和修复阶段,然后是测试——测试在编码完成之后但在下一次迭代开始之前进行。然而,真正发生的是测试被挤到迭代的末尾并且通常拖到下一个迭代中。程序员还没有太多需要解决的问题,所以他们开始着手下一次迭代。不久之后,一些团队的测试总是“落后”一个迭代,并且发布日期像往常一样被推迟。
Several people we’ve talked to described the “mini-waterfall” phenomenon that often occurs when a traditional software development organization implements an agile development process. The organization replaces a six-month or year-long development cycle with a two- or four-week one, and just tries to squeeze all of the traditional SDLC phases into that short period. Naturally, they keep having the same problems as they had before. Figure 3-1 shows an “ideal” version of the mini-waterfall where there is a code-and-fix phase and then testing—the testing comes after coding is completed but before the next iteration starts. However, what really happens is that testing gets squeezed into the end of the iteration and usually drags over into the next iteration. The programmers don’t have much to fix yet, so they start working on the next iteration. Before long, some teams are always an iteration “behind” with their testing, and release dates get postponed just as they always did.
参与交付产品的每个人都需要时间和培训来理解敏捷背后的概念以及核心实践。经验丰富的教练可用于对团队的新实践进行实践培训,例如测试驱动开发。在较大的组织中,职能测试经理可以成为实践主管,并可以提供支持和资源,以便测试人员学习如何与他们的新团队进行沟通和协作。程序员和其他团队成员需要来自职能经理的类似帮助。强大的领导力将帮助团队找到从“迷你瀑布”迁移到真正的协作开发的方法,其中编码和测试集成到一个过程中。
Everyone involved with delivering the product needs time and training to understand the concepts behind agile as well as the core practices. Experienced coaches can be used to give hands-on training in practices new to the team, such as test-driven development. In larger organizations, functional test managers can become practice leads and can provide support and resources so that testers learn how to communicate and collaborate with their new teams. Programmers and other team members need similar help from their functional managers. Strong leadership will help teams find ways to migrate away from “mini-waterfall” to true collaborative development, where coding and testing are integrated into one process.
XP 开发了一个雷达图来帮助团队确定他们对关键 XP 实践的适应程度。他们测量了五种不同的关键实践:团队、编程、规划、客户和结对,并显示了团队对实践的适应程度。图 3-2显示了两个这样的图表。图表左图显示适配成功,而右图显示存在一些问题区域。
XP has developed a radar chart to help teams determine their level of adaptation to key XP practices. They measure five different key practices: team, programming, planning, customer, and pairing, and they show the level of adaptation to practices by teams. Figure 3-2 shows two such charts. The chart on the left shows successful adaptation, while the chart on the right shows that there are some problem areas.
很多人都经历过没有坚持下去的改变。一些开发组织经历了一系列“流行方法论”。他们举起手来想:“我们为什么要再做一次?” 人们陷入了旧的、不成功的模式。即使当他们尝试新事物时,他们也可能在压力下恢复旧习惯。以下只是一些人因过去的经验和他们对“事物的现状”的看法而拒绝改变的几个例子:
Lots of people have been through changes that didn’t stick. Some development organizations have lived through a succession of the “methodology du jour.” They throw up their hands and wonder, “Why should we do it again?” People get stuck in their old, unsuccessful patterns. Even when they try something new, they might revert to bad old habits when under stress. The following are just a few examples of people resisting change due to past experience and their perception of “the way things are”:
测试人员坐在他的立方体中,不会与程序员谈论他遇到的问题。他抱怨程序员不明白他想要什么。
A tester sat in his cube and wouldn’t talk with the programmers about problems he was having. He complained that programmers didn’t understand what he wanted.
测试人员无法摆脱他现有的态度,即程序员不知道如何编写好的代码,或者如何测试它。他居高临下的态度有目共睹,他作为测试人员的信誉受到了挑战。
A tester couldn’t shake his existing attitude that programmers didn’t know how to write good code, or how to test it. His condescending attitude was clear to all, and his credibility as a tester was challenged.
当程序员做了他不喜欢的事情时,一位客户举起了手,因为他们“总是”做他们想做的事。
A customer threw up his hands when the programmers did something he didn’t like, because they “always” do what they want anyhow.
当面临向敏捷开发的过渡时,像这样的人往往不给新流程机会就离开。敏捷开发并不适合所有人,但培训和试验时间可以帮助调整态度。要求每个人都成为解决方案的一部分,并共同努力找出最适合他们特定情况的流程和实践。自组织团队可以成为一个强大的工具,可以用来让开发团队的所有成员确信他们可以控制自己的命运。
When faced with a transition to agile development, people like this often leave without giving the new process a chance. Agile development isn’t for everyone, but training and time to experiment can help adjust attitudes. Ask everyone to be part of the solution, and work together to find out what processes and practices work best for their particular situations. The self-organizing team can be a powerful tool to use to reassure all members of the development team that they’re in control of their own destiny.
每个新的敏捷团队成员都在从不同的角度进行转变。程序员通常习惯于编写产品代码并尽快发布。系统管理员和数据库专家可能习惯于在自己的孤岛中工作,按照自己的时间表执行请求。客户可能从未直接与开发团队成员交谈过。测试人员可能习惯于在项目结束时进来,根本不与程序员进行太多交互。
Each new agile team member is making the transition from a different perspective. Programmers are often used to writing production code and getting it released as quickly as possible. System administrators and database experts might be accustomed to working in their own silo, performing requests on their own schedule. Customers may never have talked directly with development team members. Testers might be used to coming in at the end of the project and not interacting much at all with programmers.
难怪向敏捷过渡会令人恐惧。团队可以制定规则和指南来帮助他们更好地沟通和合作。为了例如,Lisa 加入了一个新的敏捷团队,该团队的规则是如果有人要你与她结对,你必须同意。你可能无法在那一刻做对,但一旦你能解放自己,你就必须去帮助你的队友。
It’s no wonder a transition to agile can be scary. Teams can come up with rules and guidelines to help them communicate and work well together. For example, Lisa joined a new agile team whose rule was that if someone asked you to pair with her, you had to agree. You might not be able to do it right that minute, but as soon as you could free yourself up, you had to go help your teammate.
确定从事不同活动的人们需要什么,并找到提供这些需求的方法。客户需要某种方式来了解开发的进展情况以及是否满足他们的满意条件。开发人员需要了解业务优先级和要求。测试人员需要获取示例并将其转化为测试的方法。所有团队成员都希望感到自己是有价值的、一流的团队成员。每个团队成员还需要有安全感,可以自由地提出问题和尝试新想法。了解每个角色的观点有助于团队完成过渡。
Identify what people doing different activities need, and find ways to provide it. Customers need some way to know how development is progressing and whether their conditions of satisfaction are being met. Developers need to know business priorities and requirements. Testers need ways to capture examples and turn them into tests. All team members want to feel they are valued, first-class team members. Each team member also needs to feel safe and to feel free to raise issues and try new ideas. Understanding the viewpoint of each role helps teams through the transition.
实施任何更改时,请注意副作用。第一阶段可能是混乱;您的团队不确定新流程是什么,有些团队忠于旧方法,有些人不确定且具有破坏性。人们将这个混乱的阶段误认为是新的现状。为避免这种情况,请预先解释变更模型并设定期望。在实施敏捷流程时,期待并接受感知到的混乱。找到最痛苦的领域,并确定哪些做法可以解决问题,这样您就可以从混乱中取得一些立竿见影的进展。
When implementing any change, be aware of the side effects. The first stage may be chaos; your team isn’t sure what the new processes are, some groups are loyal to old ways, and some people are unsure and disruptive. People mistake this chaotic stage for the new status quo. To avoid this, explain the change model up front and set expectations. Expect and accept perceived chaos as you implement agile processes. Find the areas of the most pain, and determine what practices will solve the problem so that you can get some immediate progress out of the chaos.
当您开始迭代开发时,使用回顾为人们提供一个谈论他们恐惧的地方以及一个他们可以提供反馈的地方。让人们知道害怕是正常的。开放;教他们说他们害怕或不舒服是可以接受的。讨论每个恐惧的来源,从讨论中学习,做出决定,然后继续前进。恐惧是对变化的普遍反应。强迫人们做他们不想做的事情不利于积极的改变。以身作则。
When you start iterative development, use retrospectives to provide people with a place to talk about their fears and a place in which they can give feedback. Let people know that it’s normal to be fearful. Be open; teach them it is acceptable to say they are fearful or uncomfortable. Discuss each source of fear, learn from the discussion, make decisions, and move on. Fear is a common response to change. Forcing people to do something they don’t want is detrimental to positive change. Lead by example.
一个关键的成功因素是团队是否拥有所有权并有能力定制其方法。如果给予正确的帮助,人们可以改变他们的态度和看法。丽莎能够观察迈克·科恩 (Mike Cohn) 作为教练与她的团队一起工作。作为一个自组织团队,该团队必须找出并解决自己的问题。迈克确保他们有时间和资源来试验和改进。他确保企业了解质量比数量或速度更重要。每个团队,即使是自组织团队,都需要一位能够与组织的管理团队有效互动的领导者。
A critical success factor is whether the team takes ownership and has the ability to customize its approach. People can change their attitudes and their perceptions if they are given the right help. Lisa was able to observe Mike Cohn work with her team as a coach. As a self-organizing team, the team had to identify and solve its own problems. Mike made sure they had the time and resources to experiment and improve. He made sure that the business understood that quality was more important than quantity or speed. Every team, even a self-organizing team, needs a leader who can effectively interact with the organization’s management team.
实施变革需要时间并且可能令人沮丧,因此请务必庆祝您的团队取得的所有成功。当您在迭代的第四天达到为所有故事编写高级测试用例的目标时,请拍拍自己的背。当您刚刚交付了一个迭代的工作量时,召集团队一起玩问答游戏或共进午餐。如果你想让改变坚持下去,承认是很重要的。
Implementing change takes time and can be frustrating, so be sure to celebrate all successes your team achieves. Pat yourselves on the back when you meet your goal to write high-level test cases for all stories by the fourth day of the iteration. Get the team together for a trivia game or lunch when you’ve just delivered an iteration’s worth of work. Acknowledgment is important if you want a change to stick.
第 18 章,“编码和测试”,介绍了测试人员和程序员如何在整个开发过程中协同工作。
Chapter 18, “Coding and Testing,” covers how testers and programmers work together throughout the development process.
将测试人员集成到开发团队中,同时让他们继续向提供支持的 QA 经理报告是简化向敏捷开发过渡的一种方法。测试人员可以找到从与程序员的敌对关系转变为协作关系的方法。他们可以展示他们如何提供帮助团队了解客户的需求并提供适当的商业价值。他们可以举办愉快的活动来建立良好的团队互动。为队友准备饼干或巧克力是让他们走到你办公桌前的好方法!耐心和乐趣是很大的优势。
Integrating testers into development teams while letting them continue to report to a supportive QA manager is one way to ease the transition to agile development. Testers can find ways to move from an adversarial relationship with programmers to a collaborative one. They can show how they can help the team understand the customers’ needs and deliver appropriate business value. They can host enjoyable activities to build good team interactions. Having cookies or chocolate available for teammates is a good way to get them to walk over to your desk! Patience and a sense of fun are big advantages.
当我们想到采用敏捷所涉及的挑战时,我们通常会想到实际的团队及其遇到的问题。然而,要成功采用敏捷,管理层的支持至关重要。在分阶段项目中,管理层会定期获得更新和签署文件,指示每个阶段的结束。上层管理人员可能不了解他们将如何衡量敏捷项目的进度。他们可能害怕失去控制或缺乏“过程”。
When we think of challenges involved with adopting agile, we generally think of the actual team and the issues it encounters. However, for successful agile adoption, management buy-in is critical. In a phased project, management gets regular updates and sign-off documents indicating the end of each phase. Upper-level managers might not understand how they’ll be able to gauge agile project progress. They might fear a loss of control or lack of “process.”
在敏捷项目中,期望会发生变化。在她以前的瀑布项目中,Janet 记得几周以来一直听到诸如“此功能已完成 90%”之类的评论。这些类型的指标在敏捷项目中毫无意义。没有签字来标记一个阶段的结束,并且项目的“完成”不是由门来衡量的。
In an agile project, expectations change. In her previous life in waterfall projects, Janet remembers hearing comments like “this feature is 90% done” for weeks. Those types of metrics are meaningless in agile projects. There are no sign-offs to mark the end of a phase, and the “doneness” of a project isn’t measured by gates.
有意义的指标由每个项目团队确定。在 Scrum 中,冲刺和发布燃尽图跟踪故事的完成情况,并可以为经理提供进度衡量标准,但不是用于向客户开账单的任何硬性“日期”。测试矩阵可用于跟踪功能测试覆盖率,但不提供签核文档。
Meaningful metrics are determined by each project team. In Scrum, sprint and release burndown charts track story completion and can give managers a measure of progress, but not any hard “dates” to use for billing customers. Test matrices can be used to track functionality test coverage but do not provide sign-off documentation.
一些经理难以理解的另一个变化是让团队做出自己的技术决策并管理自己的工作量。不再是经理决定什么是足够好的。团队(包括客户)定义了交付成功应用程序所需的质量水平。
The other change that is difficult for some managers to understand is letting the teams make their own technical decisions and manage their own workloads. It’s no longer the manager who decides what is good enough. It is the team (which includes the customer) that defines the level of quality necessary to deliver a successful application.
敏捷团队比传统团队估计和工作的时间更短。团队需要为良好的设计和执行计划足够的时间,而不是建立应急计划,以确保技术债务不会增加。敏捷团队的管理者不是在低层级管理团队的活动,而是专注于消除障碍,以便团队成员能够做到最好。
Agile teams estimate and work in smaller chunks of time than traditional teams. Rather than building in contingency, teams need to plan enough time for good design and execution in order to ensure that technical debt does not increase. Rather than managing the team’s activities at a low level, managers of agile teams focus on removing obstacles so that team members can do their best work.
业务利益相关者不喜欢惊喜。如果可以说服他们给团队足够的时间和资源来进行过渡,他们会发现敏捷开发可以让他们更准确地规划并以稳定的增量实现业务目标。
Business stakeholders don’t like surprises. If they can be convinced to give the team enough time and resources to make the transition, they’ll find that agile development lets them plan more accurately and achieve business goals in steady increments.
有时实际上是管理层决定开始进行敏捷开发。Lisa 公司的业务负责人选择尝试敏捷开发,以解决其软件危机。为了有效,他们需要有一套不同的管理期望。他们需要对做出重大改变的困难保持敏感,尤其是在一个运作不佳的组织中。
Sometimes it’s actually management that drives the decision to start doing agile development. The business leaders at Lisa’s company chose to try agile development in order to solve its software crisis. To be effective, they needed to have a different set of management expectations. They needed to be sensitive to the difficulty of making big changes, especially in an organization that wasn’t functioning well.
在所有情况下,在向高效敏捷团队过渡的漫长过程中,管理者都需要很大的耐心。他们的工作是确保他们提供必要的资源,并确保每个人都能学习如何完成高质量的工作。
In all cases, managers need lots of patience during what might be a long transition to a high-functioning agile team. It’s their job to make sure they provide the necessary resources and that they enable every individual to learn how to do high-quality work.
如果您是 QA 经理,请准备好帮助您的测试人员克服从定义的、顺序的测试阶段转移到他们在任何一天执行广泛不同任务的快节奏迭代时的挫败感。帮助他们适应这样的想法,即测试不再是开发之后发生的单独活动,而是测试和编码是集成的活动。
If you’re a QA manager, be prepared to help your testers overcome their frustrations with moving from defined, sequential testing stages to fast-paced iterations where they perform widely varied tasks on any given day. Help them adapt to the idea that testing is no longer a separate activity that occurs after development but that testing and coding are integrated activities.
如果您是一名测试人员或其他团队成员,在向敏捷开发过渡的过程中没有获得所需的支持,请考虑您的经理在理解敏捷开发方面可能遇到的困难。帮助他们了解您需要什么样的支持。
If you’re a tester or other team member who isn’t getting the support you need in your transition to agile development, think about the difficulties your managers might be having in understanding agile development. Help them to understand what kinds of support you need.
企业管理者最了解什么?这是底线——ROI (投资回报率)。要从您的管理层那里获得您需要的支持,请在他们可以理解的背景下构建您的需求。您团队的速度转化为新功能,使业务更有利可图。如果您需要时间和资金来学习和实施自动化测试工具,请向管理层解释随着时间的推移,自动化回归测试将使您的团队更快地进行并在每次迭代中提供更多功能。
What do business managers understand best? It’s the bottom line—the ROI (return on investment). To get the support you need from your management, frame your needs in a context that they can understand. Your team’s velocity translates into new features to make the business more profitable. If you need time and funds to learn and implement an automated test tool, explain to management that over time, automated regression tests will let your team go faster and deliver more functionality in each iteration.
与敏捷团队的所有成员一样,管理人员需要学习很多新概念并弄清楚他们如何适应团队成员。使用大的可见图表(或根据需要使用它们的虚拟等价物)以确保它们可以跟踪每次迭代和发布的进度。寻找最大化投资回报率的方法。通常,当存在可提供类似价值的更简单、更快速的解决方案时,企业会要求提供复杂且昂贵的功能。确保你解释了你的团队的工作如何影响底线。与他们合作,为利益相关者找到表达每个新功能需求的最佳方式。
Like all members of an agile team, managers need to learn a lot of new concepts and figure out how they fit as team members. Use big visible charts (or their virtual equivalents, as needed) to make sure they can follow the progress of each iteration and release. Look for ways to maximize ROI. Often, the business will ask for a complex and expensive feature when there is a simpler and quicker solution that delivers similar value. Make sure you explain how your team’s work affects the bottom line. Collaborate with them to find the best way for stakeholders to express the requirements for each new feature.
预算限制是大多数团队面临的现实。当资源有限时,您的团队需要更具创造力。整个团队的方法会有所帮助。也许,像 Lisa 的团队一样,您的团队购买软件的预算有限,所以您倾向于查看通常没有大量前期购买成本的开源测试自动化工具。与应用程序使用相同语言的工具不会帮助非编程测试人员,除非程序员与他们协作以自动化测试。利用团队的所有专业知识帮助您在业务限制范围内工作。
Budget limitations are a reality most teams face. When resources are limited, your team needs to be more creative. The whole-team approach helps. Perhaps, like Lisa’s team, your team has a limited budget to buy software, and so you tend to look at open-source test automation tools that usually don’t have a large up-front purchase cost. A tool that uses the same language as the application won’t help the non-programming testers unless the programmers collaborate with them to automate the tests. Leveraging all of the expertise on the team helps you work within the business limitations.
与您的团队遇到的所有挑战一样,尝试开发团队和管理层可以互相帮助以构建有价值的产品的新方法。同时,无论您采用何种开发方法,您可能都必须确保某些过程(如符合审计要求)得到必要的关注。
As with all challenges your team encounters, experiment with new ways that the development team and management can help each other to build a valuable product. At the same time, regardless of your development approach, you might have to make sure that some processes, such as conformance to audit requirements, receive the necessary attention.
敏捷开发可能看起来节奏很快,但变化似乎很缓慢。刚接触敏捷的团队在掌握他们承诺使用的一些实践方面会很慢。我们遇到过许多测试人员,他们对他们的“敏捷”开发周期实际上是迷你瀑布周期感到沮丧。这些测试人员仍然受到挤压;它只是更频繁地发生。在可以测试故事之前,迭代就结束了。程序员拒绝或无法采用 TDD 或结对等关键实践。团队将质量责任交给测试人员,他们无力更改流程。
Agile development might seem fast-paced, but change can seem glacial. Teams that are new to agile will be slow to master some practices they’ve committed to using. We’ve met many testers who are frustrated that their “agile” development cycles are actually mini-waterfall cycles. These testers are still getting squeezed; it just happens more often. Iterations are over before stories can be tested. Programmers refuse or aren’t able to adopt critical practices such as TDD or pairing. The team leaves responsibility for quality in the hands of the testers, who are powerless to make changes to the process.
没有任何魔法可以让您的团队做出积极的改变,但我们为希望让他们的团队以积极的方式改变的测试人员提供了一些技巧。
There’s no magic that you can use to get your team to make positive changes, but we have some tips for testers who want to get their teams to change in positive ways.
TDD 等新技能很难。想方设法帮助您的团队腾出时间来掌握它们。在等待期间找到您可以独立进行的更改。例如,当程序员学习编写单元测试时,实现一个 GUI 测试工具,您只需很少的帮助即可使用。帮助团队迈出第一步。请记住,当人们恐慌时,他们会回到旧习惯,即使这些习惯不起作用。专注于微小的积极增量。
New skills such as TDD are hard. Find ways to help your team get time to master them. Find changes you can make independently while you wait. For example, while programmers learn to write unit tests, implement a GUI test tool that you can use with minimal help. Help the team make baby steps. Remember that when people panic, they go back to their old habits, even though those habits didn’t work. Focus on tiny positive increments.
有时你只需要看火车残骸。如果您的改进建议遭到拒绝,并且团队失败了,请再次提出您的建议并要求团队考虑尝试几次迭代。人们最愿意在他们感到最痛苦的地方做出改变。
Sometimes you just have to watch the train wreck. If your suggestions for improvement were rebuffed, and the team fails, bring your suggestion up again and ask the team to consider trying it for a few iterations. People are most willing to change in the areas where they feel the most pain.
您现在可能正在与以前没有与测试人员密切合作的程序员一起工作。向他们展示您可以提供哪些帮助。带着你发现的问题去找他们,而不是打开错误报告。请他们在签入之前与您一起检查代码。当他们意识到您贡献了真正的价值时,他们就更有可能倾听您的想法。
You might now be working with programmers who haven’t worked closely with testers before. Show them how you can help. Go to them with issues you’ve found rather than opening bug reports. Ask them to review code with you before they check it in. When they realize you’re contributing real value, they’re more likely to listen to your ideas.
阅读书籍和文章,参加用户组会议,学习新工具或脚本语言。开始学习您的应用程序的编码语言,并询问您团队中的程序员是否可以与他们结对或者他们是否会辅导您。您的同事会尊重您提高技能的愿望。如果您的本地用户组愿意听您关于敏捷测试的演讲,或者软件通讯发表了您的自动化文章,您的队友可能会注意到您也有一些值得一听的东西。
Read books and articles, go to user group meetings and conferences, and learn a new tool or scripting language. Start learning the language your application is coded in, and ask the programmers on your team if you can pair with them or if they’ll tutor you. Your coworkers will respect your desire to improve your skills. If your local user group is willing to listen to your presentation on agile testing, or a software newsletter publishes your automation article, your teammates might notice you have something worth hearing too.
成为合作者,而不是执行者。如果程序员不遵循编码标准,您可能会感到烦恼,但确保他们这样做不是您的工作。向团队提出您的问题并寻求他们的帮助。如果他们忽视了一个真正伤害团队的关键问题,你可能需要向你的教练或经理寻求帮助。但是要以“请帮我找到解决方案”的方式来做,而不是“让这些人表现得好”。如果您看到一个问题,其他人也很可能看到它。
Be a collaborator, not an enforcer. It might bug you if programmers don’t follow coding standards, but it’s not your job to make sure that they do so. Raise your issues with the team and ask for their help. If they ignore a critical problem that is really hurting the team, you might need to go to your coach or manager for help. But do that in a “please help me find a solution” vein rather than a “make these people behave” one. If you’re seeing a problem, chances are high that others see it too.
你一直很有耐心。你能想到的方法你都试过了,但是你的管理层不懂敏捷开发。程序员仍然会“翻墙”抛出有缺陷的、无法测试的代码,尽管你尽了最大努力,包括每天工作 14 小时,但这些代码仍按原样发布。没有人关心质量,尽管你尽了最大努力,你还是觉得自己被忽视了。也许是时候寻找一个更好的团队了。有些团队对他们现在的样子很满意,只是因为没有足够的痛苦而不想改变。丽莎所在的团队在混乱中茁壮成长,因为经常有机会找出服务器崩溃的原因并成为英雄。尽管使用敏捷实践成功完成了一个项目,但他们又回到了老习惯,Lisa 最终放弃了改变他们的尝试。
You’ve been patient. You’ve tried every approach you can think of, but your management doesn’t understand agile development. The programmers still throw buggy, untestable code “over the wall,” and that code is released as is despite your best efforts, including working 14-hour days. Nobody cares about quality, and you feel invisible despite your best efforts. It might be time to look for a better team. Some teams are happy the way they are and simply don’t feel enough pain to want to change. Lisa worked on a team that thrived on chaos, because there were frequent opportunities to figure out why the server crashed and be a hero. Despite a successful project using agile practices, they went back to their old habits, and Lisa finally gave up trying to change them.
在本章中,我们讨论了文化问题如何影响测试人员及其团队能否成功过渡到敏捷开发。
In this chapter, we talked about how cultural issues can affect whether testers and their teams can make a successful transition to doing agile development.
在做出任何改变之前考虑组织文化。
Consider organizational culture before making any kind of change.
当整个组织都重视质量时,测试人员更容易融入敏捷团队,但具有“质量警察”心态的测试人员会遇到困难。
Testers have an easier time integrating into agile teams when their whole organization values quality, but testers with a “quality police” mind-set will struggle.
一些测试人员可能难以适应“整个团队”的质量所有权,但团队方法有助于克服文化差异。
Some testers might have trouble adjusting to the “whole team” ownership of quality, but a team approach helps overcome cultural differences.
客户团队和开发团队必须紧密合作,我们展示了测试人员如何成为促进这种关系的关键。
Customer teams and developer teams must work closely together, and we showed how testers can be key in facilitating this relationship.
往往拥有更多孤立专家团队的大型组织在沟通和协作等领域面临着特殊的文化挑战。
Large organizations that tend to have more isolated specialist teams face particular cultural challenges in areas such as communication and collaboration.
测试人员成功采用敏捷的主要障碍包括恐惧、失去身份、缺乏培训、以前对新开发过程的负面体验以及角色之间的文化差异。
Major barriers to success for testers for agile adoption include fear, loss of identity, lack of training, previous negative experiences with new development processes, and cultural differences among roles.
为了帮助引入变革和促进沟通,我们建议鼓励团队成员讨论恐惧并庆祝每一次成功,无论多么小。
To help introduce change and promote communication, we suggest encouraging team members to discuss fears and celebrating every success, no matter how small.
诸如“测试人员权利法案”之类的指南让测试人员有信心提出问题,并帮助他们在学习和尝试新想法时感到安全。
Guidelines such as a “Tester Bill of Rights” give testers confidence to raise issues and help them feel safe as they learn and try new ideas.
经理们面临着他们自己的文化挑战,他们需要提供支持和培训来帮助测试人员在敏捷团队中取得成功。
Managers face their own cultural challenges, and they need to provide support and training to help testers succeed on agile teams.
测试人员可以通过提供经理跟踪进度和确定 ROI 所需的信息来帮助团队满足经理的期望。
Testers can help teams accommodate manager expectations by providing the information managers need to track progress and determine ROI.
改变来之不易,所以要有耐心,并努力提高自己的技能,这样才能帮助您的团队。
Change doesn’t come easy, so be patient, and work on improving your own skills so you can help your team.
敏捷团队强调面对面的沟通对于项目的成功至关重要。他们还鼓励使用“全团队”方法。这对测试人员意味着什么?本章讨论了一些涉及团队结构和物流的问题。创建一个有凝聚力的团队不仅仅是移动椅子和桌子。
Agile teams stress that face-to-face communication is critical to the success of a project. They also encourage using the “whole-team” approach. What does this mean to the testers? This chapter talks about some of the issues involving team structure and physical logistics. There’s more to creating a cohesive team than just moving chairs and desks.
拥有独立的职能部门会使敏捷团队的日子不好过。持续的沟通至关重要。团队成员需要彼此密切合作,无论工作是在虚拟环境中还是在同一地点完成。
Having separate functional groups can make life difficult for agile teams. Constant communication is critical. Team members need to work closely with one another, whether the work is done virtually or in the same physical location.
我们在这里互换使用术语“QA 团队”和“测试团队”。可以争论“QA 团队”是否真的在做质量保证,但这个术语已经成为测试团队的常用词,所以我们也使用它。
We use the terms “QA team” and “test team” interchangeably here. It can be argued whether “QA teams” are really doing quality assurance or not, but the term has become a common one attached to test teams, so we use it too.
许多组织,无论大小,都认为拥有一个独立的 QA 或测试团队很重要,这样才能获得关于产品质量的诚实意见。我们经常被问到这样的问题,“在整个团队方法中是否有测试组织的位置?” 和“如果是这样,它的作用是什么?”
Many organizations, both large and small, think it is important to have an independent QA or test team in order to get an honest opinion about the quality of a product. We’re often asked the questions, “Is there a place for a test organization in the whole-team approach?” and “If so, what is its role?”
我们希望将 QA 团队与开发团队分开的一些原因是:
Some of the reasons we’re given for wanting to keep the QA team separate from the development team are:
拥有独立的检查和审计角色很重要。
It is important to have that independent check and audit role.
该团队可以提供与产品质量相关的公正和外部观点。
The team can provide an unbiased and outside view relating to the quality of the product.
如果测试人员与开发人员的合作过于紧密,他们将开始像开发人员一样思考,并失去他们的客户观点。
If testers work too closely with developers, they will start to think like developers and lose their customer viewpoint.
如果测试人员和开发人员向同一个人报告,则存在优先交付任何代码而不是交付测试代码的危险。
If the testers and developers report to the same person, there is a danger that the priority becomes delivering any code rather than delivering tested code.
团队经常将“独立”与“独立”混淆。如果报告结构、预算和流程保持在独立的功能区域,则程序员和测试人员之间的划分是不可避免的。这会导致摩擦、竞争和“我们对他们”的态度。时间浪费在重复的会议上,程序员和测试人员没有共同的目标,并且不存在信息共享。
Teams often confuse “independent” with “separate.” If the reporting structure, budgets, and processes are kept in discrete functional areas, a division between the programmers and testers is inevitable. This can lead to friction, competition, and an “us versus them” attitude. Time is wasted on duplicate meetings, programmers and testers don’t share a common goal, and information sharing is nonexistent.
有一个 QA 经理和一个独立的测试团队是有原因的。但是,我们建议更改原因和结构。与其让测试人员作为一个独立的团队在编码后测试应用程序,不如将团队视为一个测试人员社区。提供一个学习型组织来帮助您的测试人员进行职业发展,并提供一个分享想法和互相帮助的地方。如果 QA 经理成为组织中的实践领导者,该人将能够教授测试人员所需的技能,使他们变得更强大,能够更好地应对不断变化的环境。
There are reasons for having a QA manager and an independent test team. However, we suggest changing the reasons as well as the structure. Rather than keeping the testers separate as an independent team to test the application after coding, think about the team as a community of testers. Provide a learning organization to help your testers with career development and a place to share ideas and help each other. If the QA manager becomes a practice leader in the organization, that person will be able to teach the skills that testers need to become stronger and better able to cope with the ever-changing environment.
我们不认为将测试人员与项目团队集成会妨碍测试人员做好他们的工作。事实上,敏捷团队的测试人员对他们作为客户倡导者的角色感觉非常强烈,也觉得他们可以影响团队的其他成员进行质量思考。
We don’t believe that integrating the testers with the project teams prevents testers from doing their jobs well. In fact, testers on agile teams feel very strongly about their role as customer advocate and also feel they can influence the rest of the team in quality thinking.
敏捷开发中的全团队方法已经促使许多采用敏捷开发的组织解散了他们独立的 QA 团队,并将他们的测试人员派往项目组工作。虽然这听起来不错,但一些组织发现它并没有按预期工作。当发现自己在一个敏捷开发团队中不知道自己应该做什么时,不止一个组织的大部分(如果不是全部)测试人员都辞职了。
The whole-team approach in agile development has provoked many organizations that have adopted agile development to disband their independent QA teams and send their testers to work with the project groups. While this sounds great, some organizations have found that it doesn’t work as expected. More than one organization has had most, if not all, of their testers quit when they found themselves on an agile development team with no idea what they should be doing.
开发人员接受结对编程、测试驱动开发和其他敏捷实践方面的培训,而测试人员通常似乎根本没有接受任何培训。许多组织未能认识到测试人员还需要接受结对测试、处理不完整和不断变化的需求、自动化以及所有其他所需的新技能方面的培训。测试人员接受培训和指导非常重要,这样他们可以获得有助于他们成功的技能和理解,例如如何与客户合作编写面向业务的测试。程序员可能还需要指导以了解面向业务的测试的重要性以及整个团队编写和自动化测试的方法。
Developers get training on pair programming, test-driven development, and other agile practices, while testers often seem to get no training at all. Many organizations fail to recognize that testers also need training on pair testing, working with incomplete and changing requirements, automation, and all of the other new skills that are required. It’s critical that testers receive training and coaching so that they can acquire the skills and understanding that will help them succeed, such as how to work with customers to write business-facing tests. Programmers also might need coaching to understand the importance of business-facing tests and the whole-team approach to writing and automating tests.
Janet 帮助将几个独立的测试团队集成到敏捷项目中。她发现大多数测试人员可能需要长达六个月的时间才能开始对使用新流程充满信心。
Janet has helped integrate several independent test teams into agile projects. She finds that it can take up to six months for most testers to start feeling confident about working with the new process.
程序员和测试人员的配对只能改善关于产品质量的沟通。如果无法在开发环境中重现该行为,开发人员通常需要在测试人员的工作站上观察应用程序的行为。测试人员有时可以与开发人员坐下来重现问题,这比他们尝试记录缺陷中的步骤更容易、更快速。这种互动减少了花在非口头交流上的时间。
The pairing of programmers and testers can only improve communication about the quality of the product. Developers often need to observe the behavior of the application on the tester’s workstation if that behavior can’t be reproduced in the development environment. Testers can sometimes sit down with the developer to reproduce a problem more easily and quickly than they can by trying to record the steps in a defect. This interaction reduces the time spent on non-oral communication.
我们从测试人员那里听到的关于此主题的评论包括以下内容:
Comments we’ve heard from testers on this subject include the following:
“更接近产品的开发让我成为更好的测试人员。”
“Being closer to the development of the product makes me a better tester.”
“与开发人员共进午餐可以建立一个更好的团队,一个希望并喜欢一起工作的团队。”
“Going to lunch with developers builds a better team, one that wants and likes to work together.”
集成项目团队的一大优势是只有一个预算和一个时间表。如果所有功能都没有完成,就没有“测试”时间可以缩短。如果没有时间测试新功能,那么一开始就没有时间开发它。正如我们在整本书中指出的那样,整个团队对质量负责的方法非常强大。
One major advantage of an integrated project team is that there’s only one budget and one schedule. There is no “testing” time to cut if all of the functionality is not finished. If there is no time to test a new feature, then there is no time to develop it in the first place. The whole-team approach to taking responsibility for quality is very powerful, as we point out throughout this book.
测试人员需要是开发团队的正式成员,测试任务需要与其他任务一样受到重视。同样,整个团队的测试方法对确保在每次迭代和发布结束时完成测试任务大有帮助。一定要使用回顾来评估测试人员需要与他们的新敏捷团队集成的内容以及他们可能需要获得的技能。例如,测试人员可能需要来自程序员或特定类型测试专家的更多支持。
Testers need to be full-fledged members of the development team, and testing tasks need to be given the same attention as other tasks. Again, the whole-team approach to testing goes a long way toward ensuring that testing tasks are completed by the end of each iteration and release. Be sure to use retrospectives to evaluate what testers need to integrate with their new agile team and what skills they might need to acquire. For example, testers might need more support from programmers, or from someone who’s an expert in a particular type of testing.
为敏捷开发规划组织变革的明智方法对成功转型至关重要。让 QA 和开发经理弄清楚他们自己在新的敏捷组织中的角色。让他们计划如何帮助他们的测试人员和开发人员在新的敏捷团队中提高工作效率。提供团队不知道的敏捷实践培训。确保所有团队都可以相互沟通。提供一个框架,让每个团队边走边学,团队将找到成功的方法。
A smart approach to planning the organizational changes for agile development makes all the difference to a successful transition. Ask the QA and development managers to figure out their own roles in the new agile organization. Let them plan how they will help their testers and developers be productive on the new agile teams. Provide training in agile practices that the team doesn’t know. Make sure all of the teams can communicate with each other. Provide a framework that lets each team learn as it goes, and the teams will find a way to succeed.
敏捷项目团队通常被认为是跨职能的,因为每个团队都有来自许多不同背景的成员。传统的跨职能团队和敏捷团队之间的区别在于整个团队努力的方法。只要项目或永久团队存在,成员就不仅仅是“代表”他们在团队中的职能,而是成为团队的真正成员(见图4-1)。
Agile project teams are generally considered cross-functional, because each team has members from many different backgrounds. The difference between a traditional cross-functional team and an agile team is the approach to the whole-team effort. Members are not just “representing” their functions in the team but are becoming true members of the team for as long as the project or permanent team exists (see Figure 4-1).
由于项目规模不同,项目团队的结构也可能不同。拥有大型项目或许多同时发生的项目的组织正在使用矩阵型结构取得成功。来自不同职能领域的人联合起来组成一个虚拟团队,同时仍然向各自的组织结构汇报。在大型组织中,一群测试人员可能会从一个项目转移到另一个项目。一些专家,例如安全或性能测试人员,可能会在多个团队之间共享。如果您正在启动一个项目,请确定该项目将需要的所有资源。在开始之前确定所需的测试人员数量和所需的技能组合。测试人员从团队开始,一直工作到项目完成,然后再继续下一个项目。
Because projects vary in size, project teams might vary in structure. Organizations with large projects or many projects that happen simultaneously are having success using a matrix-type structure. People from different functional areas combine to form a virtual team while still reporting back to their individual organizational structures. In a large organization, a pool of testers might move from project to project. Some specialists, such as security or performance testers, might be shared among several teams. If you’re starting up a project, identify all of the resources the project will need. Determine the number of testers required and the skill set needed before you start. The testers start with the team and keep working until the project is complete, and at that time they go on to the next project.
虽然测试人员是团队的一部分,但他们的日常工作与项目团队的其他工作一样受到管理。测试人员可以从更大的测试人员社区中汲取新想法,其中包括大型组织中不同项目团队的测试人员。所有测试人员都可以共享知识和想法。在实施绩效审查的组织中,QA 经理(如果有的话)可能会推动审查并从项目团队那里获得输入。
While testers are part of the team, their day-to-day work is managed the same as the rest of the project team’s work. A tester can bounce new ideas off of the larger tester community, which includes testers on different project teams across a large organization. All testers can share knowledge and ideas. In organizations that practice performance reviews, the QA manager (if there is one) might drive the reviews and get input from the project team.
与任何新团队一样,团队凝聚起来需要一段时间。如果项目的长度很短并且团队不断变化,组织需要意识到每个项目的第一次或两次迭代将包括新团队成员习惯彼此合作。根据需要重构您的组织,并记住将您的客户包括在内。最好的团队是那些学会一起工作并且彼此建立信任的团队。
As with any new team, it takes a while for a team to jell. If the length of the project is short and the teams are constantly changing, the organization needs to be aware that the first iteration or two of every project will include the new team members getting used to working with each other. Refactor your organization as needed, and remember to include your customers. The best teams are those that have learned to work together and have developed trust with one another.
许多正在考虑采用敏捷的组织都试图在不将团队置于开放式环境中的情况下创建项目团队。为了支持敏捷价值观和原则,当团队可以随时访问所有团队成员、轻松查看所有项目进度图表以及促进沟通的环境时,他们会更好地工作。
Many organizations that are thinking of adopting agile try to create project teams without co-locating the team in an open-plan environment. To support agile values and principles, teams work better when they have ready access to all team members, easy visibility of all project progress charts, and an environment that fosters communication.
靠近程序员的测试人员和客户可以实现必要的社区化。如果物流禁止同地办公,团队就可以发挥创造力。
Testers and customers sitting close to the programmers enable the necessary communalization. If logistics prohibit co-location, teams can be inventive.
团队规模为组织提供了不同类型的挑战。小团队意味着小区域,因此通常更容易将成员放在一起。大型团队可能分布在全球范围内,因此需要虚拟通信工具。将大型团队放在同一地点通常意味着翻新现有空间,而有些组织不愿意这样做。了解您的约束,并尝试为您的团队遇到的问题找到解决方案,而不是仅仅接受事物的“现状”。
Team size offers different types of challenges to the organization. Small teams mean small areas, so it is usually easier to co-locate members. Large teams might be spread globally, and virtual communication tools are needed. Co-locating large teams usually means renovating existing space, which some organizations are reluctant to do. Understand your constraints, and try to find solutions to the problems your team encounters rather than just accepting things as “the way it is.”
位于同一地点的团队并不总是生活在一个完美的世界中,分布式团队面临着另一组挑战。分布式团队需要技术来帮助他们沟通和协作。电话会议、视频会议、网络摄像头和即时消息是一些可以促进多个位置的团队进行实时协作的工具。
Co-located teams don’t always live in a perfect world, and distributed teams have a another set of challenges. Distributed teams need technology to help them communicate and collaborate. Teleconferencing, video conferencing, webcams, and instant messaging are some tools that can promote real-time collaboration for teams in multiple locations.
无论团队是同地还是分布式,通常都会出现关于敏捷团队需要哪些资源以及如何获取这些资源的相同问题。我们将在下一节中讨论这些。
Whether teams are co-located or distributed, the same questions usually come up about what resources are needed on an agile team and how to obtain them. We’ll discuss these in the next section.
新的敏捷团队成员和他们的经理对团队的构成有很多疑问。我们可以使用与传统项目相同的测试人员,还是需要雇用不同类型的测试人员?我们需要多少测试人员?我们需要具有其他专业技能的人吗?在本节中,我们将稍微讨论一下这些问题。
New agile team members and their managers have lots of questions about the makeup of the team. Can we use the same testers that we had with our traditional projects, or do we need to hire a different type of tester? How many testers will we need? Do we need people with other specialized skills? In this section, we talk a little about these questions.
关于测试人员数量与开发人员数量的“正确”比例,已经有很多讨论。组织已使用此比率来确定项目需要多少测试人员,以便他们可以相应地雇用。与传统项目一样,没有“正确”的比例,每个项目都需要单独评估。所需测试人员的数量会有所不同,具体取决于应用程序的复杂性、测试人员的技能组合以及所使用的工具。
There have been many discussions about the “right” ratio of the number of testers to the number of developers. This ratio has been used by organizations to determine how many testers are needed for a project so that they can hire accordingly. As with traditional projects, there is no “right” ratio, and each project needs to be evaluated on its own. The number of testers needed will vary and depends upon the complexity of the application, the skill set of the testers, and the tools used.
我们曾在测试人员与开发人员比例为 1:20 到 1:1 的团队中工作。这是我们的一些经验。
We have worked on teams with a tester-developer ratio of anywhere from 1:20 to 1:1. Here are a couple of our experiences.
团队不应关注比率,而应评估他们需要的测试技能并找到合适的资源。负责测试的团队可以不断评估自己是否具备所需的专业知识和带宽。使用回顾来确定是否存在雇佣更多测试人员可以解决的问题。
Rather than focus on a ratio, teams should evaluate the testing skills they need and find the appropriate resources. A team that takes responsibility for testing can continually evaluate whether it has the expertise and bandwidth it needs. Use retrospectives to identify whether there’s a problem that hiring more testers would solve.
正如我们在第 2 章“敏捷测试人员的十项原则”中所讨论的,有一些品质可以使测试人员适合在敏捷团队中工作。我们不想详细说明要聘请什么样的测试人员,因为每个团队的需求都不同。但是,我们确实认为态度是一个重要因素。这是一个关于 Lisa 的团队如何努力聘请新的敏捷测试人员的故事。
As we discussed in Chapter 2, “Ten Principles for Agile Testers,” there are certain qualities that make a tester suited to working on an agile team. We don’t want to go into a lot of detail about what kind of tester to hire, because every team’s need is different. However, we do believe that attitude is an important factor. Here’s a story of how Lisa’s team struggled to hire a new agile tester.
我们需要考虑的不仅仅是测试人员和程序员在团队中扮演的角色。无论您想担任什么角色,最重要的考虑因素是该人如何融入您的团队。使用敏捷的全团队方法,团队中的专家可能会被要求走出他们的专业领域并参与其他活动。每个团队成员都需要高度关注质量和交付业务价值。在扩大团队时,考虑的不仅仅是技术技能。
We need to consider more than just the roles that testers and programmers perform on the team. No matter what role you’re trying to fill, the most important consideration is how that person will fit on your team. With the agile whole-team approach, specialists on the team might be asked to step outside their areas of expertise and pitch in on other activities. Each team member needs to have a strong focus on quality and delivering business value. Consider more than just technical skills when you’re expanding your team.
我们已经就整个团队的方法谈了很多。但这样的变化并不仅仅发生。我们被问到这样的问题,“我们如何让团队凝聚起来?” 或“我们如何促进整个团队的方法?” 其中一个重要问题是:“我们如何让每个人都保持积极性并专注于实现商业价值的目标?”
We’ve talked a lot about the whole-team approach. But changes like that don’t just happen. We get asked questions like, “How do we get the team to jell?” or “How do we promote the whole-team approach?” One of the big ones is: “How do we keep everyone motivated and focused on the goal of delivering business value?”
根据我们的经验,当团队能够识别和解决他们自己的问题时,他们会取得最好的进展。如果您是一名经理,请抵制将您所有的好想法强加给团队的诱惑。有些问题,比如人事问题,最好由管理者来解决,而在需要领导的时候,也有教练需要大力鼓励和带领球队的时候。一个新的敏捷团队需要时间来学习如何确定问题的优先级和解决问题,但团队犯错误和跌跌撞撞几次也没关系。我们认为一个高效的团队必须自我成长。如果您是一名测试人员,您可以很好地帮助团队找出获得快速反馈的方法,使用回顾等实践来确定问题的优先级并解决问题,并找到帮助您的团队开发出更好软件的技术。
In our experience, teams make the best progress when they’re empowered to identify and solve their own problems. If you’re a manager, resist the temptation to impose all your good ideas on the team. There are problems, such as personnel issues, that are best solved by managers, and there are times a coach needs to provide strong encouragement and lead the team when it needs leadership. It takes time for a new agile team to learn how to prioritize and solve its problems, but it’s okay for the team to make mistakes and stumble a few times. We think a high-functioning team has to grow itself. If you’re a tester, you’re in a good position to help the team figure out ways to get fast feedback, use practices such as retrospectives to prioritize and address issues, and find the techniques that help your team produce better software.
您可能需要让其他团队参与进来,以帮助您的团队取得成功。安排会议;找到尽可能多的沟通方式。使用 Scrum of Scrums 来保持多个团队的协调,或者只是让其他团队参与进来。如果您必须聘请专家来帮助进行安全测试,请与该专家结对并尽可能多地学习,并帮助他们了解您的项目。
You might need to get other teams on board to help your team succeed. Set up meetings; find ways to communicate as much as possible. Use a Scrum of Scrums to keep multiple teams coordinated, or just get involved with the other teams. If you have to bring in an expert to help with security testing, pair with that expert and learn as much as you can, and help them learn about your project.
如果团队分散在不同的地点和时区,想办法尽可能多地进行直接沟通。也许代表来自每个团队可以每周调整一到两次他们的时间,这样他们就可以每周召开一次电话会议。尽可能打电话而不是发送电子邮件。Lisa 的团队调整了其计划会议时间,以包括一名在深夜工作的远程团队成员。他们将会议安排在他的一天与团队一天的其余时间重叠的时间。
If teams are scattered in different locations and time zones, figure out how to get as much direct communication as possible. Maybe representatives from each team can adjust their hours once or twice a week so that they can teleconference once a week. Make a phone call instead of sending an email whenever possible. Lisa’s team adjusted its planning meeting times to include a remote team member who works late at night. They schedule meetings for a time where his day overlaps with the rest of the team’s day.
第 9 章“支持团队的面向业务的测试工具包”给出了帮助远程团队协作的工具示例。
Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” gives examples of tools that help remote teams collaborate.
每个团队成员对团队都具有同等的价值。如果测试人员或任何其他团队成员感到被冷落或被低估,则整个团队的方法注定要失败。确保邀请测试人员参加所有会议。如果您是一名测试员,而有人忘记邀请您参加会议,请自己邀请。非技术测试人员可能认为他们会在设计会议上格格不入或不知所措,但有时他们会提出技术人员没有想到的好问题。
Every team member has equal value to the team. If testers or any other team members feel left out or less valued, the whole-team approach is doomed. Make sure testers are invited to all meetings. If you’re a tester and someone forgets to invite you to a meeting, invite yourself. Nontechnical testers might think they’ll be out of place or overwhelmed at a design meeting, but sometimes they ask good questions that the techies didn’t think of.
测试人员有权寻求和获得帮助。如果您是一名被自动化问题困住的测试人员,请鼓起勇气向团队成员寻求帮助。该人现在可能很忙,但他或她必须承诺在合理的时间内为您提供帮助。如果您是团队的经理或领导者,请确保这种情况正在发生,如果没有发生,请向团队提出问题。
Testers have a right to ask for and get help. If you’re a tester stuck on an automation problem, have the courage to ask a team member for help. That person might be busy right now, but he or she must commit to helping you in a reasonable amount of time. If you’re a manager or leader on your team, make sure this is happening, and raise the issue to the team if it’s not.
以个人为基础衡量和评估绩效可能会破坏团队协作。我们不希望程序员觉得她不应该承担测试任务,因为她被评为交付生产代码。我们不希望系统管理员忙于确保实现她的个人目标,以致于无法帮助解决测试环境问题。
Measuring and rating performance on an individual basis risks undermining team collaboration. We don’t want a programmer to feel she shouldn’t take on a testing task because she’s rated on delivering production code. We don’t want a system administrator to be so busy making sure her individual goals are met that she can’t help with a test environment problem.
相反,一个试图与团队良好合作的优秀执行者不应该因为团队的其他成员没有齐心协力而被淘汰。这是经理需要挺身而出帮助团队找到方向的时候。如果主要错误进入生产,没有人应该责怪测试人员。相反,整个团队应该分析发生的事情并开始采取措施防止再次发生。
Conversely, a good performer who was trying to work well with the team shouldn’t be knocked because the rest of the team didn’t pull together. This is a time when a manager needs to step up and help the team find its way. If major bugs made it to production, nobody should blame the testers. Instead, the whole team should analyze what happened and start taking steps to prevent a recurrence.
开发团队需要牢记业务需求。设定服务于业务的目标,提高盈利能力,让客户更快乐。与企业密切合作,让您的成功帮助整个公司取得成功。
The development team needs to keep the business needs in mind. Set goals that serve the business, increase profitability, and make the customers happier. Work closely with the business so that your successes help the whole company succeed.
正如我们在第 3 章“文化挑战”中提到的,庆祝每一次成功,无论多么小。庆祝活动可能是击掌、公司提供的午餐,或者可能只是提前下班参加一些社交活动。Lisa 团队的 ScrumMaster 在站立会议上为特殊成就颁发金星。感谢帮助您和您的团队的人。
As we mentioned in Chapter 3, “Cultural Challenges,” celebrate every success, however small. A celebration might be a high-five, a company-provided lunch, or maybe just leaving work early to socialize a bit. The ScrumMaster on Lisa’s team hands out gold stars at stand-up meetings for special accomplishments. Acknowledge the people who help you and your team.
团队可以找到新颖的方式来认可彼此的贡献。开发团队和客户团队都出席的迭代审查和演示会议是认可个人和团队成就的良好场所。
Teams can find novel ways to recognize each other’s contributions. Iteration review and demonstration meetings, where both the development team and customer team are present, are a good setting for recognizing both individual and team achievements.
阅读第 19 章“结束迭代”中的“Shout-Out Shoebox ”想法。
Read about the “Shout-Out Shoebox” idea in Chapter 19, “Wrap Up the Iteration.”
如果您是敏捷团队的新测试人员,尤其是新的敏捷团队,您可以做些什么来帮助团队克服组织挑战并取得成功?您如何融入团队并贡献您的特殊技能和经验?
If you’re a new tester on an agile team, especially a new agile team, what can you do to help the team overcome organizational challenges and succeed? How can you fit in with the team and contribute your particular skills and experience?
将我们在第 2 章中描述的十项原则付诸实践。勇气尤为重要。起身去与人交谈;询问你能提供什么帮助。通过直接沟通联系团队成员和其他团队。注意障碍并要求团队帮助移除它们。
Put the ten principles we described in Chapter 2 to work. Courage is especially important. Get up and go talk to people; ask how you can help. Reach out to team members and other teams with direct communication. Notice impediments and ask the team to help remove them.
敏捷开发之所以有效,是因为它消除了我们前进道路上的障碍,让我们做到最好。无论是个人还是团队,我们都可以感到自豪和满足。当我们遵循敏捷原则时,我们可以很好地协作,使用反馈来帮助改进我们的工作方式,并始终寻找新的更好的方法来实现我们的目标。所有这些意味着我们可以不断提高产品质量。
Agile development works because it gets obstacles out of our path and lets us do our best work. We can feel proud and satisfied, individually and as a team. When we follow agile principles, we collaborate well, use feedback to help improve how we work, and always look for new and better ways to accomplish our goals. All this means we can continually improve the quality of our product.
在本章中,我们研究了为成功的敏捷测试和开发构建团队和结构的方法。
In this chapter, we looked at ways to build a team and a structure for successful agile testing and development.
考虑团队结构的重要性;虽然测试人员可能需要独立的思维模式,但将他们放在一个单独的团队中可能会适得其反。
Consider the importance of team structure; while testers might need an independent mind-set, putting them on a separate team can be counterproductive.
测试人员需要访问更大的测试人员社区来学习和尝试新想法。QA 团队可能能够在他们的组织内创建这个社区。
Testers need access to a larger community of testers for learning and trying out new ideas. QA teams might be able to create this community within their organization.
整个团队聚在一起以促进协作很重要;如果团队是分布式的,提供促进沟通的工具。
It is important for the whole team to be located together, to foster collaboration; if the team is distributed, provide tools to promote communication.
雇用态度。
Hire for attitude.
没有正确的测试人员-开发人员比例。正确的答案是,“这取决于你的情况。”
There is no right tester–developer ratio. The right answer is, “It depends on your situation.”
团队需要自我组织,识别并找到解决自身问题的方法,并寻找改进方法。他们等不及有人告诉他们该怎么做。
Teams need to self-organize, identify and find solutions to their own problems, and look for ways to improve. They can’t wait for someone to tell them what to do.
管理层应该以促进团队努力实现业务价值的方式来奖励绩效,而不是在团队苦苦挣扎时惩罚良好的个人绩效。
Management should reward performance in a way that promotes the team’s effort to deliver business value but not penalize good individual performance if the team is struggling.
测试人员可以使用敏捷原则来提高自己的技能并增加他们对团队的价值。他们需要积极主动并找到可以做出贡献的方式。
Testers can use agile principles to improve their own skills and increase their value to the team. They need to be proactive and find ways that they can contribute.
传统项目中有许多流程不能很好地过渡到敏捷,因为它们需要重量级文档,或者是分阶段和封闭流程的固有部分,并且需要在每个阶段结束时签字。
There are many processes in a traditional project that don’t transition well to agile because they require heavyweight documentation or are an inherent part of the phased and gated process and require sign-offs at the end of each stage.
与其他任何事情一样,没有硬性规定可以将您的流程转换为更敏捷或更轻量级的流程。在本章中,我们将讨论其中的一些流程,并为您提供备选方案和指导,指导您如何在敏捷项目中使用它们。您将在第 III 部分、第 IV部分和第 V 部分中找到有关这些备选方案的更多示例和详细信息。
Like anything else, there are no hard and fast rules for transitioning your processes to a more agile or lightweight process. In this chapter, we discuss a few of those processes, and give you alternatives and guidance on how to work with them in an agile project. You’ll find more examples and details about these alternatives in Parts III, IV, and V.
当团队学习如何使用敏捷流程时,一些更传统的流程可能会在混乱中丢失。大多数习惯于工作的测试人员使用传统的分阶段和门控开发方法的人习惯于生成和使用指标,在正式的缺陷跟踪系统中记录缺陷,并编写详细的测试计划。这些在敏捷开发中适合什么地方?
When teams are learning how to use agile processes, some of the more traditional processes can be lost in the shuffle. Most testers who are used to working with traditional phased and gated development methodologies are accustomed to producing and using metrics, recording defects in a formal defect tracking system, and writing detailed test plans. Where do those fit in agile development?
许多软件组织必须遵守审计系统或质量过程模型。这些要求通常不会因为您开始使用敏捷开发实践而消失。事实上,有人担心敏捷开发会与CMMI、ISO 9000等模型和标准不兼容。
Many software organizations must comply with audit systems or quality process models. Those requirements don’t usually disappear just because you start using agile development practices. In fact, some people worry that agile development will be incompatible with such models and standards as CMMI and ISO 9000.
在敏捷项目上进行测试时,谈论所有新的和不同的东西可能会更有趣,但我们仍然需要衡量进度、跟踪缺陷和计划测试的方法。我们还需要准备好使用我们组织的质量模型。关键是让这些流程足够轻便,以帮助我们及时交付价值。让我们从查看指标开始。
It might be more fun to talk about everything that’s new and different when testing on an agile project, but we still need ways to measure progress, track defects, and plan testing. We also need to be prepared to work with our organization’s quality models. The key is to keep these processes lightweight enough to help us deliver value in a timely manner. Let’s start by looking at metrics.
指标可能会引起争议,我们花了很多时间来讨论它们。指标可能是一个浪费精力的坑,为了数字而数字。它们有时会以有害的方式使用,尽管它们不一定是坏的。他们可以指导您的团队并帮助它衡量您的团队在实现目标方面的进展。让我们来看看如何使用指标来帮助敏捷测试人员及其团队。
Metrics can be controversial, and we spend a lot of time talking about them. Metrics can be a pit of wasted effort, numbers for the sake of numbers. They are sometimes used in harmful ways, although they don’t have to be bad. They can guide your team and help it to measure your team’s progress toward its goals. Let’s take a look at how to use metrics to help agile testers and their teams.
精益软件开发从业者寻找减少测量数量的方法,并找到将驱动正确行为的测量。实施精益软件开发:从概念到现金,作者 Mary 和 Tom Poppendieck,是一个很好的资源,可以教授如何将精益计划应用于您的测试和开发工作。
Lean software development practitioners look for ways to reduce the number of measurements and find measurements that will drive the right behaviors. Implementing Lean Software Development: From Concept to Cash, by Mary and Tom Poppendieck, is an excellent resource that teaches how to apply lean initiatives to your testing and development efforts.
根据 Poppendiecks [2007] 的说法,一个基本的精益衡量标准是“从概念到现金”,从客户的功能请求到交付软件所花费的时间。他们称这种测量为“周期时间”。重点在于团队“反复且可靠地”交付新业务价值的能力。然后团队尝试不断改进他们的流程并减少周期时间。
According to the Poppendiecks [2007], a fundamental lean measurement is the time it takes to go “from concept to cash,” from a customer’s feature request to delivered software. They call this measurement “cycle time.” The focus is on the team’s ability to “repeatedly and reliably” deliver new business value. Then the team tries to continuously improve their process and reduce the cycle time.
涉及整个团队的周期时间等度量比仅限于孤立角色或团体。修复缺陷通常需要多长时间?团队可以做些什么来减少延迟,减少所花费的时间?这些类型的指标鼓励协作以进行改进。
Measurements such as cycle time that involve the whole team are more likely to drive you toward success than are measures confined to isolated roles or groups. How long does it usually take to fix a defect? What can the team do to reduce that latency, the amount of time it takes? These types of metrics encourage collaboration in order to make improvements.
Poppendiecks 在他们的书中解释的另一个精益衡量标准是财务回报。如果团队正在开发一种有利可图的产品,则需要了解如何才能实现最大利润。即使团队正在开发主要目标不是利润的内部软件或其他产品,它仍然需要查看 ROI 以确保它提供最佳价值。确定业务目标并找到衡量团队交付成果的方法。公司是否正在努力吸引新客户?跟踪新功能发布时有多少新帐户登录。
Another lean measurement the Poppendiecks explain in their book is financial return. If the team is developing a profitable product, it needs to understand how it can work to achieve the most profit. Even if the team is developing internal software or some other product whose main goal isn’t profit, it still needs to look at ROI to make sure it is delivering the best value. Identify the business goals and find ways to measure what the team delivers. Is the company trying to attract new customers? Keep track of how many new accounts sign on as new features are released.
精益开发寻找取悦客户的方法,这应该是所有软件开发的目标。Poppendiecks 举例说明了衡量客户是否满意的简单方法。
Lean development looks for ways to delight customers, which ought to be the goal for all software development. The Poppendiecks give examples of simple ways you can measure whether your customers are delighted.
我们喜欢精益指标,因为它们与我们提供商业价值的目标一致。为什么我们对指标感兴趣?我们将在下一节中进行介绍。
We like the lean metrics, because they’re congruent with our goal to deliver business value. Why are we interested in metrics at all? We’ll go into that in the next section.
收集和跟踪指标是有充分理由的。也有一些非常糟糕的。任何人都可能以糟糕的方式使用好的指标,例如将它们用作单个团队成员绩效评估的基础。然而,没有指标,你如何衡量你的进步?
There are good reasons to collect and track metrics. There are some really bad ones too. Anyone can use good metrics in terrible ways, such as using them as the basis for an individual team member’s performance evaluation. However, without metrics, how do you measure your progress?
当指标被用作路标时——告诉团队何时偏离轨道或提供反馈表明它在正确的轨道上——它们值得收集。我们的单元测试数量每天都在增加吗?为什么代码覆盖率从 75% 下降到 65%?这可能是一个很好的理由——也许我们摆脱了测试覆盖的未使用代码。指标可以提醒我们注意问题,但孤立地看,它们通常不会提供价值。
When metrics are used as guideposts—telling the team when it’s getting off track or providing feedback that it’s on the right track—they’re worth gathering. Is our number of unit tests going up every day? Why did the code coverage take a dive from 75% to 65%? It might have been a good reason—maybe we got rid of unused code that was covered by tests. Metrics can alert us to problems, but in isolation they don’t usually provide value.
衡量实现团队目标的旅程中的里程碑的指标很有用。如果我们的目标是将单元测试代码覆盖率提高 3%,我们可能会在每次签入时运行代码覆盖率,以确保我们没有懈怠单元测试。如果我们没有达到预期的改进,找出原因比抱怨我们的奖金因此减少了多少更重要。我们不应专注于个别测量,而应专注于目标和实现该目标的趋势。
Metrics that measure milestones along a journey to achieve team goals are useful. If our goal is to increase unit test code coverage by 3%, we might run the code coverage every time we check in to make sure we didn’t slack on unit tests. If we don’t achieve the desired improvement, it’s more important to figure out why than to lament whatever amount our bonus was reduced as a result. Rather than focus on individual measurements, we should focus on the goal and the trending toward reaching that goal.
指标帮助团队(包括客户)跟踪迭代和发布或史诗中的进度。如果我们正在使用燃尽图,并且我们正在燃烧而不是燃烧,这是停止的危险信号,看看发生了什么,并确保我们理解并解决问题。也许团队缺少有关故事的重要信息。指标(包括燃尽图)不应用作惩罚形式或责备来源。例如,诸如“为什么您的估计值太低?”之类的问题。或者“为什么你不能完成所有的故事?” 最好来自团队并表述为“为什么我们的估计如此低?” 和“为什么我们没有完成我们的故事?”
Metrics help the team, customers included, to track progress within the iteration and within the release or epic. If we’re using a burndown chart, and we’re burning up instead of down, that’s a red flag to stop, take a look at what’s happening, and make sure we understand and address the problems. Maybe the team lacked important information about a story. Metrics, including burndown charts, shouldn’t be used as a form of punishment or source of blame. For example, questions like “Why were your estimates too low?” or “Why can’t you finish all of the stories?” would be better coming from the team and phrased as “Why were our estimates so low?” and “Why didn’t we get our stories finished?”
如果使用得当,指标可以激励团队。Lisa 的团队跟踪每个构建中运行的单元测试的数量。重大里程碑——100 次测试、1000 次测试、3000 次测试——值得庆祝。每天增加那么多的单元测试是对开发和客户团队的一个很好的反馈。然而,重要的是要认识到数字本身没有任何意义。例如,测试可能写得不好,或者要有一个经过良好测试的产品,我们可能需要 10,000 个测试。数字不能孤立地发挥作用。
Metrics, used properly, can be motivating for a team. Lisa’s team tracks the number of unit tests run in each build. Big milestones—100 tests, 1000 tests, 3000 tests—are a reason to celebrate. Having that number of unit tests go up every day is a nice bit of feedback for the development and customer teams. However, it is important to recognize that the number itself means nothing. For example, the tests might be poorly written, or to have a well tested product, maybe we need 10,000 tests. Numbers don’t work in isolation.
当您试图弄清楚要衡量什么时,首先要了解您要解决的问题是什么。当你知道问题陈述时,你可以设定一个目标。这些目标需要是可衡量的。“将 XYZ 应用程序的平均响应时间减少到 20 个并发用户的 1.5 秒”比“提高 XYZ 应用程序性能”效果更好。如果您的目标是可衡量的,那么您需要收集的衡量指标来跟踪指标将是显而易见的。
When you’re trying to figure out what to measure, first understand what problem you are trying to solve. When you know the problem statement, you can set a goal. These goals need to be measurable. “Reduce average response time on the XYZ application to 1.5 seconds with 20 concurrent users” works better than “Improve the XYZ application performance.” If your goals are measurable, the measurements you need to gather to track the metrics will be obvious.
请记住将指标用作激励力量,而不是用来打击团队的士气。这个智慧值得重复:关注目标,而不是指标。也许您没有使用正确的指标来衡量您是否实现了团队的目标,或者您可能没有在上下文中解释它们。缺陷报告数量的增加可能意味着团队在测试方面做得更好,而不是他们编写了更多有缺陷的代码。如果你的指标无法帮助您了解实现目标的进度,您可能有错误的指标。
Remember to use metrics as a motivating force and not for beating down a team’s morale. This wisdom bears repeating: Focus on the goal, not the metrics. Maybe you’re not using the right metrics to measure whether you’re achieving your team’s objectives, or perhaps you’re not interpreting them in context. An increased number of defect reports might mean the team is doing a better job of testing, not that they are writing more buggy code. If your metrics aren’t helping you to understand your progress toward your goal, you might have the wrong metrics.
马克吐温推广了他归功于本杰明迪斯雷利的一句话:“谎言分三种:谎言、该死的谎言和统计数据。” 可衡量的目标是一件好事;如果你不能以某种方式衡量它们,你就无法判断你是否实现了它们。另一方面,使用指标来判断个人或团队的表现是危险的。统计数据本身可以被扭曲成任何解释并以有害的方式使用。
Mark Twain popularized the saying, which he attributed to Benjamin Disraeli, “There are three kinds of lies: lies, damned lies, and statistics.” Measurable goals are a good thing; if you can’t gauge them in some way, you can’t tell if you achieved them. On the other hand, using metrics to judge individual or team performance is dangerous. Statistics by themselves can be twisted into any interpretation and used in detrimental ways.
以代码行为例,这是一种传统的软件量尺。更多的代码行是好事,意味着团队一直在生产,还是坏事,意味着团队正在编写低效的意大利面条式代码?
Take lines of code, a traditional software measuring stick. Are more lines of code a good thing, meaning the team has been productive, or a bad thing, meaning the team is writing inefficient spaghetti-style code?
发现的缺陷数量如何?根据测试人员发现的缺陷数量来判断测试人员是否有意义?这如何帮助他们更好地完成工作?可以肯定地说,每行代码产生更多缺陷的开发团队做得不好吗?或者发现更多缺陷的团队做得很好?即使这个想法站得住脚,一个团队被数字打得头破血流又有多大的动力?这会让团队成员开始编写无缺陷的代码吗?
What about number of defects found? Does it make any sense to judge testers by the number of defects they found? How does that help them do their jobs better? Is it safe to say that a development team that produces a higher number of defects per lines of code is doing a bad job? Or that a team that finds more defects is doing a good job? Even if that thought holds up, how motivating is it for a team to be whacked over the head with numbers? Will that make the team members start writing defect-free code?
我们知道,无论我们衡量什么,都必然会发生变化。有多少测试正在运行并通过?距离我们需要“现成的构建”还有多少天?完整构建是否通过?我们看不到和容易解释的指标不值得拥有。如果您想跟踪通过测试的数量,请确保该指标以正确的方式对正确的人可见。可见的大图表是我们所知的显示指标的最有效方式。
We know that whatever we measure is bound to change. How many tests are running and passing? How many days until we need a “build on the shelf”? Is the full build passing? Metrics we can’t see and easily interpret aren’t worth having. If you want to track the number of passing tests, make sure that metric is visible in the right way, to the right people. Big visible charts are the most effective way of displaying metrics we know.
你的指标值得麻烦吗?不要为了产生数字而测量。想想你将从这些数字中学到什么。在下一节中,我们将考虑您可以从指标中获得的投资回报。
Are your metrics worth the trouble? Don’t measure for the sake of producing numbers. Think about what you’ll learn from those numbers. In the next section, we consider the return on investment you can expect from metrics.
当您确定所需的指标时,请确保您可以以合理的成本获得它们。如果您的持续构建提供了有用的数字,那么它会带来很好的价值。无论如何,您都在运行构建,如果它给我们提供了额外的信息,那就太好了。如果你需要做很多额外的工作来获取信息,问问自己这是否值得。
When you identify the metrics you need, make sure you can obtain them at a reasonable cost. If your continual build delivers useful numbers, it delivers good value. You’re running the build anyway, and if it gives us extra information, that’s gravy. If you need a lot of extra work to get information, ask yourself if it’s worth the trouble.
Lisa 的团队在跟踪每个故事实际花费的时间与估计时间方面遇到了很多麻烦。除了显而易见的事实,他们还学到了什么那估计只是这样吗?不多。一些有经验的团队发现他们可以省去冲刺燃尽图,因为任务板给了他们足够的信息来衡量他们的进度。他们可以将花费的时间用于估计任务并计算剩余时间以进行更有成效的活动。
Lisa’s team went to a fair amount of trouble to track actual time spent per story versus estimated time. What did they learn other than the obvious fact that estimates are just that? Not much. Some experienced teams find they can dispense with the sprint burndown chart because the task board gives them enough information to gauge their progress. They can use the time spent estimating tasks and calculating the remaining hours on more productive activities.
这并不意味着我们建议您停止跟踪这些测量值。新团队需要了解他们的速度和燃尽率,这样他们才能稳步提高。
This doesn’t mean we recommend that you stop tracking these measurements. New teams need to understand their velocity and burndown rate, so that they can steadily improve.
缺陷率是传统的软件指标,对于以零缺陷为目标的团队而言,它们可能没有太大价值。了解在开发过程中发现和修复错误的比率没有多大价值,因为发现和修复错误是开发过程中不可或缺的一部分。如果测试人员向正在处理代码的程序员展示了一个缺陷,并且编写了单元测试并立即修复了错误,则通常不需要记录缺陷。另一方面,如果许多缺陷在未被发现的情况下进入生产阶段,那么跟踪这些缺陷的数量以了解团队是否有所改进是有价值的。
Defect rates are traditional software metrics, and they might not have much value on a team that’s aiming for zero defects. There’s not much value in knowing the rate of bugs found and fixed during development, because finding and fixing them is an integral part of development. If a tester shows a defect to the programmer who’s working on the code, and a unit test is written and the bug is fixed right away, there’s often no need to log a defect. On the other hand, if many defects reach production undetected, there can be value in tracking the number to know if the team improves.
当它开始重写有缺陷的遗留应用程序时,Lisa 的团队设定了一个目标,即在新代码投入生产后六个月内报告的新代码中的高严重性缺陷不超过六个。拥有一个直接且易于跟踪的目标有助于激励团队找到在开发过程中阻止错误并超越该目标的方法。
When it started to rewrite its buggy legacy application, Lisa’s team set a goal of no more than six high-severity bugs in new code reported after the code is in production over a six-month period. Having a target that was straightforward and easy to track helped motivate the team to find ways to head bugs off during development and exceed this objective.
计算每个指标的投资回报率并决定是跟踪还是维护它。收集它所花费的努力是否证明它提供的价值是合理的?是否易于沟通和理解?一如既往,做适合你情况的事情。尝试在几个冲刺中保持特定的指标,并评估它是否有效。
Figure each metric’s return on investment and decide whether to track or maintain it. Does the effort spent collecting it justify the value it delivers? Can it be easily communicated and understood? As always, do what works for your situation. Experiment with keeping a particular metric for a few sprints and evaluate whether it’s paying off.
与软件质量相关的一种常见指标是缺陷率。在下一节中,我们将探讨跟踪缺陷或不跟踪缺陷的原因,以及我们可以从中学到什么。
One common metric that relates to software quality is the defect rate. In the next section, we look at reasons to track defects, or to not track defects, and what we can learn from them.
每个新的敏捷团队都会问的一个问题是,“我们是否仍在缺陷跟踪系统中跟踪错误?” 没有简单的答案,但我们会就此事向您提供我们的意见并提供一些替代方案,以便您确定适合您的团队的方案。
One of the questions that are asked by every new agile team is, “Do we still track bugs in a defect tracking system?” There’s no simple answer, but we’ll give you our opinion on the matter and offer some alternatives so that you can determine what fits your team.
我们中的许多测试人员都将缺陷跟踪作为交流我们看到的问题的唯一方式,并且很容易继续使用我们熟悉的工具。DTS 是一个方便的地方,不仅可以跟踪缺陷,还可以跟踪优先级、严重性和状态,并查看它被分配给了谁。许多敏捷实践者说我们不需要再这样做了,我们可以通过卡片或其他一些简单的机制来跟踪缺陷。我们可以编写一个测试来显示失败,修复代码,并将测试保留在我们的回归套件中。
A lot of us testers have used defect tracking as the only way to communicate the issues we saw, and it’s easy to keep using the tools we are familiar with. A DTS is a convenient place to keep track of not only the defect but the priorities, severities, and status, and to see who it is assigned to. Many agile practitioners say that we don’t need to do this anymore, that we can track defects on cards or some other simple mechanism. We could write a test to show the failure, fix the code, and keep the test in our regression suite.
但是,有理由继续使用工具来记录缺陷及其修复方式。现在让我们探讨其中的一些。
However, there are reasons to keep using a tool to record defects and how they were fixed. Let’s explore some of them now.
不保留缺陷跟踪系统的担忧之一是没有地方保留错误的所有细节。测试人员习惯于记录带有大量信息的错误,例如如何重现它,它是在什么环境中发现的,或者使用的是什么操作系统或浏览器。所有这些信息都无法放在一张卡片上,那么您如何捕捉这些细节呢?如果你只依赖卡片,你还需要交谈。但是通过谈话,细节会丢失,有时测试人员会忘记到底做了什么——特别是如果在程序员解决问题前几天发现了错误。
One of the concerns about not keeping a defect tracking system is that there is no place to keep all of the details of the bug. Testers are used to recording a bug with lots of information, such as how to reproduce it, what environment it was found in, or what operating system or browser was used. All of this information cannot fit on a card, so how do you capture those details? If you are relying only on cards, you also need conversation. But with conversation, details get lost, and sometimes a tester forgets exactly what was done—especially if the bug was found a few days prior to the programmer tackling the issue.
DTS 也是保存所有补充文档(如屏幕打印或上传文件)的便利位置。
A DTS is also a convenient place to keep all supplemental documentation, such as screen prints or uploaded files.
我们听说过跟踪缺陷的理由,例如“我们需要能够查看旧的错误报告”。我们试图思考为什么您需要查看旧错误报告的原因,并且在我们研究本章时,珍妮特找到了一个例子。
We have heard reasons to track defects such as, “We need to be able to look at old bug reports.” We tried to think of reasons why you would ever need to look at old bug reports, and as we were working on this chapter, Janet found an example.
这个故事表明,虽然错误数据库可以用作知识库,但可能还有其他机制来保存业务决策及其背景信息。如果一个 issue 太老以至于已经忘记了,也许我们应该重写它并再次提出来。情况可能已经改变,企业可能会决定现在修复错误是值得的。
This story shows that although the bug database can be used as a knowledge base, there might be other mechanisms for keeping business decisions and their background information. If an issue is old enough to have been lost track of, maybe we should rewrite it and bring it up again. The circumstances may have changed, and the business might decide it is now worthwhile to fix the bug.
便于保留在 DTS 中的错误类型是间歇性的并且需要很长时间才能追踪到的错误。这些错误很少出现,并且通常会出现时间间隔,在此期间调查会因缺乏信息而停滞不前。DTS 是一个可以捕获有关到目前为止已计算出的信息的地方。它还可以包含日志、跟踪等。当团队中的某个人终于有时间查看问题或问题变得更加关键时,这可能是有价值的信息。
The types of bugs that are handy to keep in a DTS are the ones that are intermittent and take a long time to track down. These bugs present themselves infrequently, and there are usually gaps in time during which the investigation stalls for lack of information. A DTS is a place where information can be captured about what was figured out so far. It can also contain logs, traces, and so on. This can be valuable information when someone on the team finally has time to look at the problem or the issue becomes more critical.
错误报告中的信息以后可以用于多种目的。这是 Lisa 团队关于如何使用其信息的故事。
The information in bug reports can be used later for several purposes. Here’s a story from Lisa’s team on how it uses its information.
当团队特别大或不在同一地点时,记住缺陷的原因或为满足特殊要求所做的工作就更难了。客户也可能对其问题的解决方案感兴趣。
Remembering the cause of defects or what was done to fulfill a special request is even harder when the team is particularly large or isn’t co-located. Customers might also be interested in the solutions to their problems.
如果项目太大以至于一个团队发现的缺陷可能会影响其他团队,那么 DTS 可能是一个不错的选择。当然,要发挥作用,团队的所有成员都需要能够访问它。面对面交流始终是我们的首选,但当情况变得不切实际时,我们需要 DTS 等辅助工具。
If projects are so large that defects found by one team might affect other teams, a DTS is probably a good choice. Of course, to be useful it needs to be accessible to all members of the team. Face-to-face communication is always our first choice, but when circumstances make that impractical, we need aids such as a DTS.
当发布后有客户报告的缺陷时,客户通常想知道这些缺陷何时得到修复。对于帮助台或技术支持来说,了解给定版本中修复了什么是非常宝贵的。他们还可以找到发布时仍未解决的缺陷,并告知客户。DTS 使将这些信息集中在一起变得更加简单。
When there are defects that have been reported by the customer after the release, the customer usually wants to know when they’ve been fixed. It’s invaluable for the help desk or technical support to know what was fixed in a given release. They can also find defects that are still outstanding at release time and let the customers know. A DTS makes it much simpler to pull this information together.
跟踪缺陷率是有原因的。也有一些您不跟踪缺陷的原因。例如,我们认为如果 bug 从未脱离迭代,则不应将其视为缺陷。当然,这会引发另一个关于我们应该跟踪什么以及为什么要跟踪的讨论,但我们不会在这里讨论。
There are reasons to track defect rates. There are also reasons why you wouldn’t track a defect. For example, we don’t think that a bug should be counted as a defect if it never makes it out of the iteration. This, of course, brings up another discussion about what should we track and why, but we won’t discuss that here.
Chapter 18, “Coding and Testing,” explores metrics related to defect rates.
我们听说拥有 DTS 的另一个原因是可追溯性,将缺陷链接到测试用例。我们不确定这是一个正当理由。并非所有缺陷都与测试用例有关,也不应该与测试用例有关。例如,拼写错误之类的错误可能不需要特定的测试用例。也许该产品使用起来不直观;这是一个非常真实的错误,通常不会被报告。您如何编写测试以确定某些东西是否可用?探索性测试可能会发现边缘条件中的错误,这些错误不值得创建自动化测试。
Another reason we’ve heard for having a DTS is traceability, linking defects to test cases. We’re not sure that this is a valid reason. Not all defects are linked to test cases, nor should they be. For example, errors like spelling mistakes might not need specific test cases. Maybe the product was not intuitive to use; this is a very real bug that often goes unreported. How do you write a test to determine if something is usable? Exploratory testing might find bugs in edge conditions that are not worth the effort of creating automated tests.
如果它是一个自动测试用例捕获了一个错误,那么记录该缺陷的需要会进一步减少,因为如果重新引入它会再次被捕获。对可追溯性的需求消失了。所以,也许我们不需要跟踪缺陷。
If it is an automated test case that caught a bug, then the need to record that defect is further reduced, because it will be caught again if ever reintroduced. The need for traceability is gone. So, maybe we don’t need to track defects.
敏捷和精益为我们提供了有助于减少对 DTS 需求的实践和原则。如果过程是可靠的,并且所有人都致力于提供高质量的产品,那么缺陷应该很少见并且很容易被跟踪。
Agile and Lean provide us with practices and principles that help reduce the need for a DTS. If the process is solid, and all of the people are committed to delivering a quality product, defects should be rare and very simply tracked.
缺陷跟踪系统当然不会促进程序员和测试人员之间的交流。他们可以很容易地避免直接与对方交谈。
Defect tracking systems certainly don’t promote communication between programmers and testers. They can make it easy to avoid talking directly to each other.
除了重现缺陷的所有步骤之外,我们倾向于将大量信息放入 DTS。根据错误的不同,可能需要很长时间才能编写这些步骤,以便程序员也可以重现它。然后是分类,必须有人发表评论、解释缺陷、尝试重现它、(理想情况下)修复它、写更多评论,然后将它分配回报告它的人。最后,可以验证修复。如果程序员一开始就误解了问题,整个周期可能会加倍。单个缺陷报告的成本可能会变得过高。
We tend to put lots of information into the DTS in addition to all of the steps to reproduce the defect. Depending on the bug, it can take a long time to write these steps so that the programmer can reproduce it as well. Then there is the triage, and someone has to make comments, interpret the defect, attempt to reproduce it, (ideally) fix it, write more comments, and assign it back to the person who reported it. Finally, the fix can be verified. This whole cycle can double if the programmer misunderstood the problem in the first place. The cost of a single defect report can become exorbitant.
如果我们作为测试人员可以与程序员交谈并展示我们发现的内容,然后开发人员立即修复缺陷,那会容易得多?我们稍后会详细讨论。
How much easier would it be if we as testers could just talk to the programmer and show what we found, with the developer then fixing the defect right away? We’ll talk more about that later.
在第 15 章“编码和测试”中,我们将解释测试人员和程序员如何协同工作来解决错误。
In Chapter 15, “Coding and Testing,” we’ll explain how tester and programmers work together on bugs.
DTS 中的缺陷成为队列或迷你产品积压。根据精益原则,这种缺陷清单是一种浪费。作为一个团队,我们应该想办法减少这种浪费。
Defects in a DTS become a queue or a mini product backlog. According to lean principles, this inventory of defects is a waste. As a team, we should be thinking of ways to reduce this waste.
当我们在第 18 章“编码和测试”中介绍迭代计划时,Antony 将分享他关于隐藏积压的想法。
Antony will share his ideas about the hidden backlog when we cover iteration planning in Chapter 18, “Coding and Testing.”
如果您决定使用 DTS,请谨慎选择。了解您的需求并保持简单。您会希望团队中的每个人都使用它。如果它变得开销很大或难以使用,人们会想方设法解决它。对于敏捷开发团队使用的所有工具,您应该考虑整个团队的意见。如果客户团队的任何人输入错误报告,也请听取他或她的意见。
If you do decide to use a DTS, choose it carefully. Understand your needs and keep it simple. You will want everyone on the team to use it. If it becomes overhead or hard to use, people will find ways to work around it. As with all tools used by your agile development team, you should consider the whole team’s opinion. If anyone from the customer team enters bug reports, get his or her opinion too.
Janet 使用过的最简单的工具之一是 Alcea 的 FIT IssueTrack。它是可配置的,不会让您遵循预定义的流程,并且很容易从中获取指标。做好功课,找到适合您的工具。有多种开源缺陷跟踪系统、托管系统和集成企业系统可用。
One of the simplest tools that Janet has used is Alcea’s FIT IssueTrack. It is configurable, does not make you follow a predefined process, and is easy to get metrics out of. Do your homework and find the tool that works for you. There are a variety of open source defect-tracking systems, hosted systems, and integrated enterprise systems available.
无论您是否使用 DTS,您都希望缺陷尽可能可见。
Whether or not you use a DTS, you want to make defects as visible as possible.
我们通常建议尝试使用不同的工具,使用每一种工具进行几次迭代,但这对于错误跟踪系统来说比较棘手,因为您需要将一个系统中的所有错误移植到您正在尝试的新系统中尺寸。花一些时间思考您在 DTS 中需要什么,它将服务于什么目的,并明智地评估备选方案。
We usually recommend experimenting with different tools, using each one for a few iterations, but this is trickier with bug-tracking systems, because you need to port all of the bugs that are in one system to the new one that you’re trying on for size. Spend some time thinking about what you need in a DTS, what purposes it will serve, and evaluate alternatives judiciously.
与所有工具搜索一样,向社区中的其他人(例如用户组和邮件列表)寻求建议。在开始寻找之前定义您的标准,并尽可能多地进行实验。如果您选择了错误的工具,请减少损失并开始研究替代品。
As with all your tool searches, look to others in your community, such as user groups and mailing lists, for recommendations. Define your criteria before you start looking, and experiment as much as you can. If you choose the wrong tool, cut your losses and start researching alternatives.
关于报告和跟踪缺陷的决定很重要,但不要忘记您的主要目标。您希望尽可能提供最优质的产品,并且希望及时为企业创造价值。项目当允许人们尽力而为时,就会成功。专注于改善沟通和建立协作。如果您遇到很多缺陷,请调查问题的根源。如果您需要 DTS 来执行此操作,请使用它。如果您的团队通过记录可执行测试中的缺陷并立即修复它们来更好地工作,那就去做吧。如果某种组合能让你不断改进,那就去吧。要记住的主要事情是它必须适用于您的整个团队。
Decisions about reporting and tracking defects are important, but don’t lose track of your main target. You want to deliver the best quality product you can, and you want to deliver value to the business in a timely manner. Projects succeed when people are allowed to do their best work. Concentrate on improving communication and building collaboration. If you encounter a lot of defects, investigate the source of the problem. If you need a DTS to do that, use it. If your team works better by documenting defects in executable tests and fixing them right away, do that. If some combination enables you continually improve, go with it. The main thing to remember is that it has to work for your whole team.
第 18 章,“编码和测试”涵盖了备选方案,并向您展示了解决错误问题的不同方法。
Chapter 18, “Coding and Testing,” covers alternatives and shows you different ways to attack your bug problems.
缺陷跟踪是敏捷测试中产生最多问题和争议的典型质量流程之一。混淆的另一个重要来源是敏捷项目是否需要测试计划或可追溯性矩阵等文档。接下来让我们考虑一下。
Defect tracking is one of the typical quality processes that generate the most questions and controversy in agile testing. Another big source of confusion is whether agile projects need documents such as test plans or traceability matrices. Let’s consider that next.
传统的分阶段软件方法强调测试计划作为整体文档需求的一部分的重要性。它们旨在为涉众概述软件测试工作的目标、范围、方法和重点。完成的文档旨在帮助测试组以外的人员了解产品验证的“原因”和“方式”。在本节中,我们将研究测试计划以及准备和跟踪敏捷项目测试工作的其他方面。
Traditional phased software methodologies stress the importance of test plans as part of the overall documentation needs. They’re intended to outline the objectives, scope, approach, and focus of the software testing effort for stakeholders. The completed document is intended to help people outside the test group understand the “why” and “how” of product validation. In this section, we look at test plans and other aspects of preparing and tracking the testing effort for an agile project.
在敏捷项目中,团队不依赖大量文档来传达测试人员需要做什么。测试人员与团队的其他成员携手合作,以便测试工作以任务卡的形式对所有人可见。所以经常向我们提出的问题是,“是否还需要测试计划?” 要回答这个问题,让我们首先看一下测试计划和测试策略或方法之间的区别。
In an agile project, teams don’t rely on heavy documentation to communicate what the testers need to do. Testers work hand in hand with the rest of the team so that the testing efforts are visible to all in the form of task cards. So the question often put to us is, “Is there still a need for test plans?” To answer that question, let’s first take a look at the difference between a test plan and a test strategy or approach.
文档中包含的信息越多,有人阅读所有内容的可能性就越小。考虑利益相关者真正需要哪些信息。想想它的使用频率和用途。
The more information that is contained in a document, the less likely it is that someone is going to read it all. Consider what information is really necessary for the stakeholders. Think about how often it is used and what it is used for.
我们喜欢将测试策略视为很少更改的静态文档,而测试计划是新创建的并且特定于每个新项目。
We like to think of a test strategy as a static document that seldom changes, while a test plan is created new and is specific to each new project.
战略是一项长期的行动计划,关键词是“长期”。如果您的组织需要有关您的整体测试方法的文档项目,请考虑获取此信息并将其放入静态文档中,该文档不会随时间发生太大变化。有很多信息不是项目特定的,可以提取到测试策略或测试方法文档中。
A strategy is a long-term plan of action, the key word being “long-term.” If your organization wants documentation about your overall test approach to projects, consider taking this information and putting it in a static document that doesn’t change much over time. There is a lot of information that is not project specific and can be extracted into a Test Strategy or Test Approach document.
然后可以将此文档用作参考,并且只有在流程发生变化时才需要更新。测试策略文档可用于让新员工对您的测试过程如何工作有一个更高层次的理解。
This document can then be used as a reference and needs to be updated only if processes change. A test strategy document can be used to give new employees a high-level understanding of how your test processes work.
规划的力量在于识别可能的问题和依赖性,将风险浮出水面以供讨论和解决,并考虑大局。测试计划也不例外。团队应该在开始之前考虑每个项目的风险和依赖关系以及全局。
The power of planning is to identify possible issues and dependencies, to bring risks to the surface to be talked about and to be addressed, and to think about the big picture. Test planning is no different. A team should think about risks and dependencies and the big picture for each project before it starts.
无论您的团队是否决定创建测试计划文档,都应该进行计划。每个项目都是不同的,所以不要指望相同的解决方案适用于所有项目。
Whether your team decides to create a test plan document or not, the planning should be done. Each project is different, so don’t expect that the same solution will fit all.
在第 15 章“发布或主题规划中的测试人员活动”中,我们展示了示例并讨论了您在规划发布时可以使用的替代方案。
In Chapter 15, “Tester Activities in Release or Theme Planning,” we show examples and discuss alternatives you can use when you are planning the release.
有时我们的客户坚持要一份测试计划文件。如果您签订合同开发应用程序,则测试计划可能是一组可交付成果的一部分,这些交付成果还包括需求文档和设计文档等项目。
Sometimes our customers insist on a test plan document. If you’re contracting to develop an application, a test plan might be part of a set of deliverables that also include items such as a requirements document and a design document.
谈论测试计划通常会导致谈论可追溯性。是否有人对交付的代码执行了预期行为的所有计划测试?需求和测试计划如何与实际测试和最终功能相关联?
Talk of test plans often leads to talk of traceability. Did someone execute all planned testing of the desired behavior on the delivered code? How do requirements and test plans relate to the actual testing and final functionality?
在传统项目中,我们过去常常需要可追溯性矩阵来确定我们是否实际测试了所有需求。如果需求发生变化,我们需要知道我们已经更改了适当的测试用例。对于非常大的需求文档,这是测试团队知道它具有良好覆盖率的唯一方式。
In traditional projects, we used to need traceability matrices to determine whether we had actually tested all of the requirements. If a requirement changed, we needed to know that we had changed the appropriate test cases. With very large requirements documents, this was the only way that a test team knew it had good coverage.
在敏捷项目中,我们没有这些限制。我们以微小的、定义明确的步骤构建功能。我们与团队密切合作,知道什么时候会发生变化。如果程序员先进行测试,我们就会知道所有小块工作都有单元测试。然后我们可以与客户合作定义验收测试。我们在程序员处理每个故事时对其进行测试,因此我们知道没有什么是未经测试的。
In an agile project, we don’t have those restrictions. We build functionality in tiny, well-defined steps. We work with the team closely and know when something changes. If the programmers work test-first, we know there are unit tests for all of the small chunks of work. We can then collaborate with the customer to define acceptance tests. We test each story as the programmer works on it, so we know that nothing goes untested.
受监管行业可能需要某种可追溯性。如果有,我们建议你真正看看问题管理试图解决什么问题。当您了解需要什么时,您应该尝试使解决方案尽可能简单。有多种方法可以提供可追溯性。源代码签入评论可以参考包含需求或测试用例的 wiki 页面,或者参考缺陷号。您可以在单元测试中添加注释,将测试与需求的位置或标识符联系起来。测试可以直接与 FitNesse 等工具中的需求集成。您的团队可以轻松找到最适合客户需求的方式。
There might be requirements for some kind of traceability for regulated industries. If there is, we suggest that you really look at what problem management is trying to solve. When you understand what is needed, you should try to make the solution as simple as possible. There are multiple ways to provide traceability. Source code check-in comments can refer to the wiki page containing the requirements or test cases, or to a defect number. You can put comments in unit tests tying the test to the location or identifier of the requirement. The tests can be integrated directly with the requirements in a tool such as FitNesse. Your team can easily find the way that works best for your customers’ needs.
可能需要诸如可追溯性矩阵之类的文件来满足组织的审计标准或质量模型强加的要求。让我们考虑一下这些指令如何与敏捷开发相得益彰。
Documents such as traceability matrices might be needed to fulfill requirements imposed by the organization’s audit standards or quality models. Let’s consider how these directives get along with agile development.
这个问题经常被问到:“传统的质量模型和流程能否与敏捷开发方法共存?” 从理论上讲,他们没有理由不能。实际上,往往别无选择。质量模型通常属于传统 QA 团队的领域,他们也可以跟随测试人员进入新的敏捷结构。将它们融入新的敏捷开发模型可能并不容易。让我们看看一些典型的质量流程,以及测试人员和他们的团队如何适应这些流程。
This question is often asked: “Can traditional quality models and processes coexist with agile development methods?” In theory, there is no reason why they can’t. In reality, there is often not a choice. Quality models often fall into the domain of the traditional QA team, and they can follow testers into the new agile structure as well. It might not be easy to fit these into a new agile development model. Let’s look at a few typical quality processes and how testers and their teams might accommodate them.
不同的行业有不同的审计要求。传统开发组织中的质量保证团队通常负责为审计员提供信息并确保符合审计要求。为应对引人注目的公司财务丑闻而颁布的 2002 年萨班斯-奥克斯利法案规定了维护业务记录的要求。确保合规性通常落在 IT 部门的肩上。SAS 70 是另一个广泛认可的服务组织审计标准。这些只是影响开发团队的审计控制类型的几个示例。
Different industries have different audit requirements. Quality assurance teams in traditional development organizations are often tasked with providing information for auditors and ensuring compliance with audit requirements. The Sarbanes-Oxley Act of 2002, enacted in response to high-profile corporate financial scandals, sets out requirements for maintaining business records. Ensuring compliance usually falls to the IT departments. SAS 70 is another widely recognized auditing standard for service organizations. These are just a couple of examples of the type of audit controls that affect development teams.
较大的组织有专门的团队来控制合规性并与审计员合作,但开发团队经常被要求提供信息。示例包括在给定的软件版本上执行了哪些测试,或证明不同的帐户协调一致。测试人员的任务是编写测试计划来评估控制活动的有效性。
Larger organizations have specialized teams that control compliance and work with auditors, but development teams are often asked to provide information. Examples include what testing has been performed on a given software release, or proving that different accounts reconcile. Testers can be tasked with writing test plans to evaluate the effectiveness of control activities.
属于敏捷团队的测试人员应该致力于该团队。如果需要他们的帮助来提供审计信息或帮助确保合规性,为此写故事,并与团队的其他工作一起计划。与合规和内部审计团队合作,了解您团队的职责。
Testers who are part of an agile team should be dedicated to that team. If their help is needed in providing information for an audit or helping to ensure compliance, write stories for this and plan them along with the rest of the team’s work. Work together with the compliance and internal audit teams to understand your team’s responsibilities.
质量模型有很多,但我们只看其中两个,以展示如何调整敏捷过程以适应它们的约束。
There are many quality models, but we’ll look at two to show how you can adapt your agile process to fit within their constraints.
1.能力成熟度模型集成 (CMMI) 旨在帮助组织改进他们的过程,但并不规定具体的开发实践来完成改进。
1. The Capability Maturity Model Integration (CMMI) aims to help organizations improve their process but doesn’t dictate specific development practices to accomplish the improvements.
2.信息技术基础架构库 (ITIL) 是一组 IT 服务管理最佳实践,旨在帮助组织开发有效的质量流程。
2. Information Technology Infrastructure Library (ITIL) is a set of best practices for IT service management intended to help organizations develop an effective quality process.
这两种模型都可以与敏捷开发愉快地共存。他们植根于同一个目标,即使软件开发项目取得成功。
Both of these models can coexist happily with agile development. They’re rooted in the same goal, making software development projects succeed.
让我们看看 CMMI,这是一个用于衡量流程成熟度的框架。它通过测量过程是未知的、已定义的、记录的、永久的还是优化的来定义每个级别。敏捷项目有一个明确的过程,尽管并非所有团队都记录他们所做的事情。例如,通过发布计划墙上的索引卡管理您的需求,并由单个客户做出最终决定,只要您一直这样做,就是一个明确的过程。
Let’s look at CMMI, a framework for measuring the maturity of your process. It defines each level by measuring whether the process is unknown, defined, documented, permanent, or optimized. Agile projects have a defined process, although not all teams document what they do. For example, managing your requirements with index cards on a release planning wall with a single customer making the final decisions is a defined process as long as you do it all the time.
回顾旨在不断改进流程,团队应该始终寻找优化流程的方法。如果您的团队唯一缺乏的是文档,那么请考虑将您的过程包含到您的测试策略文档中。
Retrospectives are aimed at constant process improvement, and teams should be always be looking for ways to optimize processes. If the only thing your team is lacking is documentation, then think about including your process into your test strategy documentation.
问问自己,满足 CMMI 要求的最少文档数量是多少。Janet 在使用图 5-2中的图表方面取得了成功。
Ask yourself what the minimum amount of documentation you could give to satisfy the CMMI requirements would be. Janet has had success with using diagrams like the one in Figure 5-2.
如果 ITIL 已被引入您的组织并影响变更管理,请调整您的流程以适应它。您甚至可能会发现新流程是有益的。
If ITIL has been introduced into your organization and affects change management, adapt your process to accommodate it. You might even find the new process beneficial.
如果您的组织正在使用某种流程模型或质量标准管理,请自行学习并与组织中合适的专家合作。保持团队专注于交付可提供真正业务价值的高质量软件,并了解您如何在模型中工作。
If your organization is using some kind of process model or quality standards management, educate yourself about it, and work with the appropriate specialists in your organization. Maintain the team’s focus on delivering high-quality software that provides real business value, and see how you can work within the model.
流程改进模型和框架强调流程的纪律和一致性。很少有软件开发方法比敏捷开发需要更多的纪律。标准只是让您能够衡量您实现目标的进度。敏捷的重点是尽力而为并不断改进。敏捷开发与实现您为自己设定的任何标准或从过程改进测量工具中借用的标准兼容。
Process improvement models and frameworks emphasize discipline and conformance to process. Few software development methodologies require more discipline than agile development. Standards simply enable you to measure your progress toward your goal. Agile’s focus is on doing your best work and constantly improving. Agile development is compatible with achieving whatever standards you set for yourself or borrow from a process improvement measurement tool.
将您的测量目标和标准与改进这些测量的方法分开。设定目标,并了解您需要哪些指标来衡量需要改进的领域是否成功。尝试使用任务卡提供改进的活动,以确保他们获得所需的可见性。
Separate your measurement goals and standards from your means to improve those measurements. Set goals, and know what metrics you need to measure success for areas that need improvement. Try using task cards for activities that provide the improvements in order to ensure they get the visibility they need.
使用现有的质量流程和模型是您在过渡到敏捷开发时可能面临的最大文化问题之一。所有这些改变都是艰难的,但是当你的整个团队都参与进来时,没有什么是不可克服的。
Working with existing quality processes and models is one of the biggest cultural issues you may face as you transition to agile development. All of these changes are hard, but when your whole team gets involved, none are insurmountable.
在本章中,我们研究了传统的以质量为导向的流程,以及它们如何适应敏捷环境。
In this chapter, we looked at traditional quality-oriented processes and how they can be adapted for an agile environment.
正确的指标可以帮助您确保您的团队走上正轨以实现其目标,并为您的投资提供良好的回报。
The right metrics can help you to make sure your team is on track to achieve its goals and provide a good return on your investment in them.
指标应该是可见的,提供必要的里程碑来做出决策。
Metrics should be visible, providing necessary milestones upon which to make decisions.
使用缺陷跟踪系统的原因包括方便、用作知识库和可追溯性。
The reasons to use a defect tracking system include for convenience, for use as a knowledge base, and for traceability.
缺陷跟踪系统经常被用作通信工具,输入和跟踪不必要的错误可能被认为是一种浪费。
Defect tracking systems are too often used as a communication tool, and entering and tracking unnecessary bugs can be considered wasteful.
所有的工具,包括DTS,都是需要整个团队使用的,所以在选择工具的时候要多方面考虑。
All tools, including the DTS, need to be used by the whole team, so consider all perspectives when choosing a tool.
测试策略是一种长期的整体测试方法,可以放在静态文档中;测试计划应该是项目唯一的。
A test strategy is a long-term overall test approach that can be put in a static document; a test plan should be unique to the project.
在盲目接受对特定文件的需求之前考虑替代方案。例如,以小的增量块进行开发的敏捷方法,紧密合作,可能会消除对正式可追溯性文档的需求。将源代码控制系统注释链接到测试可能是另一种方式。
Think about alternatives before blindly accepting the need for specific documents. For example, the agile approach to developing in small, incremental chunks, working closely together, might remove the need for formal traceability documents. Linking the source code control system comments to tests might be another way.
传统的质量流程和流程改进模型,例如 SAS 70 审计和 CMMI 标准,可以与敏捷开发和测试共存。团队需要对跳出框框的思维持开放态度,并共同努力解决他们的问题。
Traditional quality processes and process improvement models, such as SAS 70 audits and CMMI standards, can coexist with agile development and testing. Teams need to be open to thinking outside the box and work together to solve their problems.
软件质量有很多维度,每个维度都需要不同的测试方法。我们怎么知道我们需要做的所有不同类型的测试?我们如何知道何时“完成”测试?谁进行哪些测试以及如何进行?在这一部分中,我们将解释如何使用敏捷测试象限来确保您的团队涵盖所有需要的测试类别。
Software quality has many dimensions, each requiring a different testing approach. How do we know all the different types of tests we need to do? How do we know when we’re “done” testing? Who does which tests and how? In this part, we explain how to use the Agile Testing Quadrants to make sure your team covers all needed categories of testing.
当然,测试需要工具,我们已经包含了使用工具的示例、有效使用这些工具的策略以及有关何时使用它们的指南。当与为可测试性而设计的代码一起使用时,工具更易于使用。本书的这一部分将讨论这些问题以及更多问题。
Of course, testing requires tools, and we’ve included examples of tools to use, strategies for using those tools effectively, and guidelines about when to use them. Tools are easier to use when used with code that’s designed for testability. These concerns and more are discussed in this part of the book.
我们为什么要测试?答案似乎很明显,但实际上非常复杂。我们出于很多原因进行测试:发现错误,确保代码可靠,有时只是为了查看代码是否可用。我们进行不同类型的测试以实现不同的目标。软件产品质量有很多组成部分。在本章中,我们将介绍敏捷测试象限。第三部分的其余章节将详细介绍每个象限。敏捷测试象限矩阵帮助测试人员确保他们已经考虑了交付价值所需的所有不同类型的测试。
Why do we test? The answer might seem obvious, but in fact, it’s pretty complex. We test for a lot of reasons: to find bugs, to make sure the code is reliable, and sometimes just to see if the code’s usable. We do different types of testing to accomplish different goals. Software product quality has many components. In this chapter, we introduce the Agile Testing Quadrants. The rest of the chapters in Part III go into detail on each of the quadrants. The Agile Testing Quadrants matrix helps testers ensure that they have considered all of the different types of tests that are needed in order to deliver value.
在第 1 章中,“到底什么是敏捷测试?”,我们针对实现不同目的的不同类别的测试介绍了 Brian Marick 的术语。图 6-1是敏捷测试象限图,显示了四个象限中的每一个如何反映我们测试的不同原因。在一个轴上,我们将矩阵分为支持团队的测试和批评产品的测试。另一个轴将它们分为面向业务和面向技术的测试。
In Chapter 1, “What Is Agile Testing, Anyway?,” we introduced Brian Marick’s terms for different categories of tests that accomplish different purposes. Figure 6-1 is a diagram of the agile testing quadrants that shows how each of the four quadrants reflects the different reasons we test. On one axis, we divide the matrix into tests that support the team and tests that critique the product. The other axis divides them into business-facing and technology-facing tests.
我们对这些象限进行编号的顺序与不同类型测试的完成时间无关。例如,敏捷开发从客户测试开始,它告诉团队编码什么。各种类型测试的时间取决于每个项目的风险、客户对产品的目标、团队是在使用遗留代码还是在新建项目上工作,以及何时有资源可用于进行测试。
The order in which we’ve numbered these quadrants has no relationship to when the different types of testing are done. For example, agile development starts with customer tests, which tell the team what to code. The timing of the various types of tests depends on the risks of each project, the customers’ goals for the product, whether the team is working with legacy code or on a greenfield project, and when resources are available to do the testing.
左侧的象限包括支持团队开发产品的测试。这种帮助程序员的测试概念对许多测试人员来说是新的,并且是传统项目测试与敏捷项目测试之间的最大区别。在象限 1 和 2 中完成的测试比我们通常认为的测试更多的是需求规范和设计辅助。
The quadrants on the left include tests that support the team as it develops the product. This concept of testing to help the programmers is new to many testers and is the biggest difference between testing on a traditional project and testing on an agile project. The testing done in Quadrants 1 and 2 are more requirements specification and design aids than what we typically think of as testing.
The lower left quadrant represents test-driven development, which is a core agile development practice.
单元测试验证系统的一小部分功能,例如对象或方法。组件测试验证系统较大部分的行为,例如一组提供某些服务的类 [ Meszaros, 2007 ]。这两种类型的测试通常都使用 xUnit 测试自动化工具家族的成员进行自动化。我们将这些测试称为程序员测试、面向开发人员的测试或面向技术的测试。它们使程序员能够衡量 Kent Beck 所说的代码的内部质量 [Beck, 2000, 2004]。
Unit tests verify functionality of a small subset of the system, such as an object or method. Component tests verify the behavior of a larger part of the system, such as a group of classes that provide some service [Meszaros, 2007]. Both types of tests are usually automated with a member of the xUnit family of test automation tools. We refer to these tests as programmer tests, developer-facing tests, or technology-facing tests. They enable the programmers to measure what Kent Beck has called the internal quality of their code [Beck, 2000, 2004].
象限 1 测试的一个主要目的是测试驱动开发 (TDD) 或测试驱动设计。编写测试的过程首先帮助程序员很好地设计他们的代码。这些测试让程序员自信地编写代码来提供故事的功能,而不必担心对系统进行意外更改。他们可以验证他们的设计和架构决策是否合适。单元和组件测试是自动化的,并使用与应用程序相同的编程语言编写。业务专家可能无法通过直接阅读来理解它们,但这些测试并非旨在供客户使用。事实上,内部质量不是与客户协商的;它是由程序员定义的。程序员测试通常是随着每次代码签入而运行的自动化过程的一部分,为团队提供即时、
A major purpose of Quadrant 1 tests is test-driven development (TDD) or test-driven design. The process of writing tests first helps programmers design their code well. These tests let the programmers confidently write code to deliver a story’s features without worrying about making unintended changes to the system. They can verify that their design and architecture decisions are appropriate. Unit and component tests are automated and written in the same programming language as the application. A business expert probably couldn’t understand them by reading them directly, but these tests aren’t intended for customer use. In fact, internal quality isn’t negotiated with the customer; it’s defined by the programmers. Programmer tests are normally part of an automated process that runs with every code check-in, giving the team instant, continual feedback about their internal quality.
象限 2 中的测试也支持开发团队的工作,但级别更高。这些面向业务的测试,也称为面向客户的测试和客户测试,定义了外部质量和客户想要的功能。
The tests in Quadrant 2 also support the work of the development team, but at a higher level. These business-facing tests, also called customer-facing tests and customer tests, define external quality and the features that the customers want.
与象限 1 测试一样,它们也推动开发,但在更高的层次上。通过敏捷开发,这些测试源自客户团队提供的示例。他们描述了每个故事的细节。面向业务的测试在功能级别运行,每个测试都验证业务满意度条件。它们是以业务专家可以使用业务领域语言轻松理解的方式编写的。事实上,业务专家使用这些测试来定义产品的外部质量,并且通常帮助编写它们。这个象限可能会重复一些在单元级别完成的测试;然而,象限 2 测试旨在说明和确认更高级别的所需系统行为。
Like the Quadrant 1 tests, they also drive development, but at a higher level. With agile development, these tests are derived from examples provided by the customer team. They describe the details of each story. Business-facing tests run at a functional level, each one verifying a business satisfaction condition. They’re written in a way business experts can easily understand using the business domain language. In fact, the business experts use these tests to define the external quality of the product and usually help to write them. It’s possible this quadrant could duplicate some of the tests that were done at the unit level; however, the Quadrant 2 tests are oriented toward illustrating and confirming desired system behavior at a higher level.
第 8 章,“支持团队的面向业务的测试”,解释了满意度的业务条件。
Chapter 8, “Business-Facing Tests that Support the Team,” explains business conditions of satisfaction.
大多数支持开发团队的面向业务的测试也需要自动化。在这两个象限中进行测试的最重要目的之一是快速提供信息并实现快速故障排除。它们必须经常运行,以便在任何行为意外变化的情况下为团队提供早期反馈。如果可能,这些自动化测试直接在生产代码中的业务逻辑上运行,而无需通过表示层。尽管如此,一些自动化测试必须验证用户界面和客户端应用程序可能使用的任何 API。所有这些测试都应该作为自动化持续集成、构建和测试过程的一部分运行。
Most of the business-facing tests that support the development team also need to be automated. One of the most important purposes of tests in these two quadrants is to provide information quickly and enable fast troubleshooting. They must be run frequently in order to give the team early feedback in case any behavior changes unexpectedly. When possible, these automated tests run directly on the business logic in the production code without having to go through a presentation layer. Still, some automated tests must verify the user interfaces and any APIs that client applications might use. All of these tests should be run as part of an automated continuous integration, build, and test process.
还有另一组测试也属于这个象限。用户交互专家使用模型和线框来帮助与客户验证提议的 GUI(图形用户界面)设计,并在开发人员开始对其进行编码之前将这些设计传达给他们。该组中的测试是帮助支持团队正确构建产品但不是自动化的测试。正如我们将在接下来的章节中看到的那样,象限帮助我们识别我们需要使用的所有不同类型的测试,以帮助推动编码。
There is another group of tests that belongs in this quadrant as well. User interaction experts use mock-ups and wireframes to help validate proposed GUI (graphical user interface) designs with customers and to communicate those designs to the developers before they start to code them. The tests in this group are tests that help support the team to get the product built right but are not automated. As we’ll see in the following chapters, the quadrants help us identify all of the different types of tests we need to use in order to help drive coding.
有些人使用术语“验收测试”来描述象限 2 测试,但我们认为验收测试包含更广泛的测试范围,包括象限 3 和 4。验收测试验证系统的所有方面,包括可用性和性能,满足客户要求。
Some people use the term “acceptance tests” to describe Quadrant 2 tests, but we believe that acceptance tests encompass a broader range of tests that include Quadrants 3 and 4. Acceptance tests verify that all aspects of the system, including qualities such as usability and performance, meet customer requirements.
象限 1 和 2 自动化测试提供的快速反馈,随着每次代码更改或添加而运行,构成了敏捷团队的基础。这些测试首先指导功能的开发,当自动化时,然后提供安全网以防止重构和新代码的引入导致意外结果。
The quick feedback provided by Quadrants 1 and 2 automated tests, which run with every code change or addition, form the foundation of an agile team. These tests first guide development of functionality, and when automated, then provide a safety net to prevent refactoring and the introduction of new code from causing unexpected results.
象限 1 和 2 中的测试旨在帮助团队交付客户要求的业务价值。他们验证业务逻辑和用户界面的行为是否符合客户提供的示例。软件质量还有其他方面,如果没有技术团队的帮助,客户不会考虑其中的一些方面。产品有竞争力吗?用户界面是否像它需要的那样直观?应用程序安全吗?用户对用户界面的工作方式是否满意?我们需要不同的测试来回答这些类型的问题。
The tests in Quadrants 1 and 2 are written to help the team deliver the business value requested by the customers. They verify that the business logic and the user interfaces behave according to the examples provided by the customers. There are other aspects to software quality, some of which the customers don’t think about without help from the technical team. Is the product competitive? Is the user interface as intuitive as it needs to be? Is the application secure? Are the users happy with how the user interface works? We need different tests to answer these types of questions.
如果您一直担任客户角色并且必须表达您对软件功能的要求,您就会知道在您看到它之前要确切知道您想要什么是多么困难。即使您对该功能的工作方式有信心,也很难描述它以使程序员完全理解它。
If you’ve been in a customer role and had to express your requirements for a software feature, you know how hard it can be to know exactly what you want until you see it. Even if you’re confident about how the feature should work, it can be hard to describe it so that programmers fully understand it.
“批评”这个词并不具有消极意义。批评可以包括表扬和改进建议。评估软件产品涉及艺术和科学。我们以建设性的方式审查软件,目的是了解我们如何改进它。随着我们的学习,我们可以将新的需求和测试或示例反馈给支持团队和指导开发的流程。
The word “critique” isn’t intended in a negative sense. A critique can include both praise and suggestions for improvement. Appraising a software product involves both art and science. We review the software in a constructive manner, with the goal of learning how we can improve it. As we learn, we can feed new requirements and tests or examples back to the process that supports the team and guide development.
面向业务的示例可以帮助团队设计所需的产品,但至少我们的一些示例可能是错误的。业务专家可能会忽略功能,或者如果这不是他们的专业领域,则不会完全正确。团队可能只是误解了一些例子。即使程序员编写了使面向业务的测试通过的代码,他们也可能无法交付客户真正想要的东西。
Business-facing examples help the team design the desired product, but at least some of our examples will probably be wrong. The business experts might overlook functionality, or not get it quite right if it isn’t their field of expertise. The team might simply misunderstand some examples. Even when the programmers write code that makes the business-facing tests pass, they might not be delivering what the customer really wants.
这就是评判第三和第四象限产品的测试发挥作用的地方。象限 3 对运行工作软件的面向业务的测试进行分类,以查看它是否不完全符合预期或无法经受住竞争。当我们进行面向业务的测试来评价产品时,我们会尝试模仿真实用户使用应用程序的方式。这是只有人类才能进行的手动测试。我们可能会使用一些自动化的脚本来帮助我们设置我们需要的数据,但我们必须用我们的感官、我们的大脑和我们的直觉来检查开发团队是否已经交付了客户所需要的商业价值。
That is where the tests to critique the product in the third and fourth quadrants come into play. Quadrant 3 classifies the business-facing tests that exercise the working software to see if it doesn’t quite meet expectations or won’t stand up to the competition. When we do business-facing tests to critique the product, we try to emulate the way a real user would work the application. This is manual testing that only a human can do. We might use some automated scripts to help us set up the data we need, but we have to use our senses, our brains, and our intuition to check whether the development team has delivered the business value required by the customers.
通常,用户和客户执行这些类型的测试。用户验收测试 (UAT) 使客户有机会很好地锻炼新功能并了解他们将来可能需要哪些更改,这是收集新故事创意的好方法。如果您的团队在合同基础上向客户交付软件,则 UAT 可能是批准已完成故事的必要步骤。
Often, the users and customers perform these types of tests. User Acceptance Testing (UAT) gives customers a chance to give new features a good workout and see what changes they may want in the future, and it’s a good way to gather new story ideas. If your team is delivering software on a contract basis to a client, UAT might be a required step in approving the finished stories.
可用性测试是一种测试类型的例子,它有自己的一整套科学。可能会引入焦点小组,在他们使用应用程序时对其进行研究,并进行访谈以收集他们的反应。可用性测试还可以包括从一个页面到另一个页面的导航,甚至是像 Tab 键顺序这样简单的东西。在测试可用性时,了解人们如何使用系统是一个优势。
Usability testing is an example of a type of testing that has a whole science of its own. Focus groups might be brought in, studied as they use the application, and interviewed in order to gather their reactions. Usability testing can also include navigation from page to page or even something as simple as the tabbing order. Knowledge of how people use systems is an advantage when testing usability.
探索性测试是该象限的核心。在探索性测试期间,测试人员同时设计和执行测试,使用批判性思维来分析结果。与脚本测试相比,这提供了更好的了解应用程序的机会。我们不是在谈论即兴和即兴的临时测试。探索性测试是一种比临时测试更周到、更复杂的方法。它以策略为指导,并在定义的约束范围内运行。从每个项目和故事开始,测试人员就开始思考他们想要尝试的场景。随着可测试代码的小块变得可用,测试人员分析测试结果,并且在他们学习的过程中,他们找到新的领域进行探索。探索性测试以与最终用户相同的方式运行系统。测试人员使用他们的创造力和直觉。
Exploratory testing is central to this quadrant. During exploratory testing sessions, the tester simultaneously designs and performs tests, using critical thinking to analyze the results. This offers a much better opportunity to learn about the application than scripted tests. We’re not talking about ad hoc testing, which is impromptu and improvised. Exploratory testing is a more thoughtful and sophisticated approach than ad hoc testing. It is guided by a strategy and operates within defined constraints. From the start of each project and story, testers start thinking of scenarios they want to try. As small chunks of testable code become available, testers analyze test results, and as they learn, they find new areas to explore. Exploratory testing works the system in the same ways that the end users will. Testers use their creativity and intuition. As a result, it is through this type of testing that many of the most serious bugs are usually found.
属于第四象限的测试类型对于敏捷开发和任何类型的软件开发一样重要。这些测试是面向技术的,我们以技术而非业务术语来讨论它们。象限 4 中面向技术的测试旨在评判产品特性,例如性能、稳健性和安全性。正如我们将在第 11 章“使用面向技术的测试来评价产品”中所述,您的团队已经具备进行这些测试所需的许多技能。例如,程序员可能能够利用多线程引擎将单元测试用于性能测试。但是,创建和运行这些测试可能需要使用专门的工具和额外的专业知识。
The types of tests that fall into the fourth quadrant are just as critical to agile development as to any type of software development. These tests are technology-facing, and we discuss them in technical rather than business terms. Technology-facing tests in Quadrant 4 are intended to critique product characteristics such as performance, robustness, and security. As we’ll describe in Chapter 11, “Critiquing the Product Using Technology-Facing Tests,” your team already possesses many of the skills needed to do these tests. For example, programmers might be able to leverage unit tests into performance tests with a multi-threaded engine. However, creating and running these tests might require the use of specialized tools and additional expertise.
过去,我们听到抱怨说敏捷开发似乎忽略了批评产品的面向技术的测试。这些抱怨可能部分是由于敏捷强调让客户编写故事并确定故事的优先级。非技术客户团队成员通常认为开发人员会处理速度和安全性等问题,而程序员只打算生产客户优先考虑的功能。
In the past, we’ve heard complaints that agile development seems to ignore the technology-facing tests that critique the product. These complaints might be partly due to agile’s emphasis on having customers write and prioritize stories. Nontechnical customer team members often assume that the developers will take care of concerns such as speed and security, and that the programmers are intent on producing only the functionality prioritized by the customers.
如果我们在开始编码之前了解性能、安全性、与其他系统的交互以及其他非功能性属性的要求,那么在设计和编码时考虑到这一点会更容易。其中一些可能比实际功能更重要。例如,如果一个 Internet 零售网站只有一分钟的响应时间,那么客户将不会等待欣赏所有功能都正常工作的事实。在开发周期的每一步都应该考虑对产品进行批评的面向技术的测试,而不是留到最后。在许多情况下,此类测试甚至应该在功能测试之前完成。
If we know the requirements for performance, security, interaction with other systems, and other nonfunctional attributes before we start coding, it’s easier to design and code with that in mind. Some of these might be more important than actual functionality. For example, if an Internet retail website has a one-minute response time, the customers won’t wait to appreciate the fact that all of the features work properly. Technology-facing tests that critique the product should be considered at every step of the development cycle and not left until the very end. In many cases, such testing should even be done before functional testing.
近年来,我们看到许多适用于敏捷开发项目的新轻量级工具可用于支持测试。自动化工具可用于创建测试数据、为手动测试设置测试场景、驱动安全测试并帮助理解结果。对于负载和性能测试等某些工作,自动化是强制性的。
In recent years we’ve seen many new lightweight tools appropriate to an agile development project become available to support tests. Automation tools can be used to create test data, set up test scenarios for manual testing, drive security tests, and help make sense of results. Automation is mandatory for some efforts such as load and performance testing.
当您和您的团队计划新版本或项目时,讨论您需要象限 3 和 4 中的哪些类型的测试,以及应该在何时完成这些测试。不要将负载或可用性测试等基本活动留到最后,否则纠正问题可能为时已晚。
When you and your team plan a new release or project, discuss which types of tests from Quadrants 3 and 4 you need, and when they should be done. Don’t leave essential activities such as load or usability testing to the end, when it might be too late to rectify problems.
在审查产品的测试过程中产生的信息应该反馈到我们矩阵的左侧,并用于创建新的测试以推动未来的发展。例如,如果服务器在正常负载下出现故障,则需要新的故事和测试来驱动更具可扩展性的架构。使用象限将帮助您计划批评产品的测试以及推动开发的测试。想想你为什么要测试,以确保测试是在开发的最佳阶段进行的。
The information produced during testing to review the product should be fed back into the left side of our matrix and used to create new tests to drive future development. For example, if the server fails under a normal load, new stories and tests to drive a more scalable architecture will be needed. Using the quadrants will help you plan tests that critique the product as well as tests that drive development. Think about why you are testing to make sure that the tests are performed at the optimum stage of development.
敏捷开发的短期迭代让您的团队有机会学习和试验不同的测试象限。如果您发现您的设计无法扩展为时已晚,请尽早开始对下一个故事或项目进行负载测试。如果迭代演示表明团队误解了客户的需求,那么您可能没有做好编写客户测试来指导开发的工作。如果团队推迟了必要的重构,也许单元和组件测试没有提供足够的覆盖率。使用敏捷测试象限来帮助确保在正确的时间完成所有必要的测试。
The short iterations of agile development give your team a chance to learn and experiment with the different testing quadrants. If you find out too late that your design doesn’t scale, start load testing earlier with the next story or project. If the iteration demo reveals that the team misunderstood the customer’s requirements, maybe you’re not doing a good enough job of writing customer tests to guide development. If the team puts off needed refactoring, maybe the unit and component tests aren’t providing enough coverage. Use the agile testing quadrants to help make sure all necessary testing is done at the right time.
对于大多数产品,我们需要进行所有四类测试才能确信我们提供了正确的价值。并非每个故事都需要进行安全测试,但您不会因为没有想到而忽略它。
For most products, we need all four categories of testing to feel confident we’re delivering the right value. Not every story requires security testing, but you don’t want to omit it because you didn’t think of it.
驱动开发的面向技术和面向业务的测试是敏捷开发的核心,无论您是否真的为它们编写任务卡。它们为您的团队提供了“完成”每个故事的最佳机会。确定执行评估产品的面向技术和面向业务的测试所需的任务可确保您了解产品缺少的内容。来自所有四个象限的测试组合将使团队知道每个特性何时满足客户的功能和质量标准。
The technology-facing and business-facing tests that drive development are central to agile development, whether or not you actually write task cards for them. They give your team the best chance of getting each story “done.” Identifying the tasks needed to perform the technology-facing and business-facing tests that critique the product ensures that you’ll learn what the product is missing. A combination of tests from all four quadrants will let the team know when each feature has met the customer’s criteria for functionality and quality.
我们的产品团队需要广泛的专业知识来涵盖所有敏捷测试象限。程序员应该编写支持编程的面向技术的测试,但他们可能在不同时间需要测试人员、数据库设计人员、系统管理员和配置专家的帮助。测试人员与客户一起主要负责面向业务的测试,但程序员参与设计和自动化测试,同时可能会根据需要召集可用性专家和其他专家。第四象限,通过面向技术的测试来评判产品,可能需要更多的专家。无论必须从开发团队外部引入什么资源,该团队仍然负责完成所有四个象限的测试。
Our product teams need a wide range of expertise to cover all of the agile testing quadrants. Programmers should write the technology-facing tests that support programming, but they might need help at different times from testers, database designers, system administrators, and configuration specialists. Testers take primary charge of the business-facing tests in tandem with the customers, but programmers participate in designing and automating tests, while usability and other experts might be called in as needed. The fourth quadrant, with technology-facing tests that critique the product, may require more specialists. No matter what resources have to be brought in from outside the development team, the team is still responsible for getting all four quadrants of testing done.
我们相信,一个成功的团队是每个人都参与产品的制作,并且当出现问题时每个人都分担团队内部的痛苦。实施使我们能够的实践和工具解决所有四个象限的测试有时会很痛苦,但实现成功产品的乐趣是值得付出努力的。
We believe that a successful team is one where everybody participates in the crafting of the product and that everyone shares the team’s internal pain when things go wrong. Implementing the practices and tools that enable us to address all four quadrants of testing can be painful at times, but the joy of implementing a successful product is worth the effort.
Ward Cunningham 于 1992 年创造了“技术债务”一词,但我们在软件开发的整个职业生涯中肯定都经历过它!当开发团队走捷径、快速修复或者跳过编写或自动化测试时,技术债务就会增加,因为它处于枪口之下。代码库变得越来越难维护。与金融债务一样,“利息”以更高的维护成本和更低的团队速度的形式复合。程序员害怕做任何改变,更不用说尝试重构来改进代码了,因为害怕破坏它。有时,这种恐惧的存在是因为他们一开始无法理解编码,有时是因为没有测试来发现错误。
Ward Cunningham coined the term “technical debt” in 1992, but we’ve certainly experienced it throughout our careers in software development! Technical debt builds up when the development team takes shortcuts, hacks in quick fixes, or skips writing or automating tests because it’s under the gun. The code base gets harder and harder to maintain. Like financial debt, “interest” compounds in the form of higher maintenance costs and lower team velocity. Programmers are afraid to make any changes, much less attempt refactoring to improve the code, for fear of breaking it. Sometimes this fear exists because they can’t understand the coding to start with, and sometimes it is because there are no tests to catch mistakes.
敏捷测试矩阵中的每个象限都在将技术债务保持在可管理的水平方面发挥作用。支持编码和设计的面向技术的测试有助于保持代码的可维护性。运行单元测试的自动化构建和集成过程是最小化技术债务的必要条件。在编码期间捕获单元级缺陷将使测试人员能够专注于面向业务的测试,以指导团队和改进产品。及时的负载和压力测试让团队知道他们的架构是否能胜任工作。
Each quadrant in the agile testing matrix plays a role in keeping technical debt to a manageable level. Technology-facing tests that support coding and design help keep code maintainable. An automated build and integration process that runs unit tests is a must for minimizing technical debt. Catching unit-level defects during coding will free testers to focus on business-facing tests in order to guide the team and improve the product. Timely load and stress testing lets the teams know whether their architecture is up to the job.
通过花时间并应用资源和实践来将技术债务保持在最低限度,团队将有时间和资源来完成确保产品质量所需的测试。应用敏捷原则在每个级别做好每种类型的测试,反过来又会最大限度地减少技术债务。
By taking the time and applying resources and practices to keep technical debt to a minimum, a team will have time and resources to cover the testing needed to ensure a quality product. Applying agile principles to do a good job of each type of testing at each level will, in turn, minimize technical debt.
我们在敏捷测试矩阵中找到的分类和定义帮助我们确保我们计划并完成我们需要的所有不同类型的测试。但是,我们需要记住,每个组织、产品和团队都有自己独特的情况,每个人都需要根据自己的情况做适合自己的事情。正如 Lisa 的同事 Mike Busse 常说的那样,“这是一种工具,而不是规则。” 随着时间的推移,单个产品或项目的需求可能会发生巨大变化。象限是确保您的团队正在考虑进入“完成”的测试的所有不同方面的有用方法。
Categorizations and definitions such as we find in the agile testing matrix help us make sure we plan for and accomplish all of the different types of testing we need. However, we need to bear in mind that each organization, product, and team has its own unique situation, and each needs to do what works for it in its individual situation. As Lisa’s coworker Mike Busse likes to say, “It’s a tool, not a rule.” A single product or project’s needs might evolve drastically over time. The quadrants are a helpful way to make sure your team is considering all of the different aspects of testing that go into “doneness.”
在为每个故事、迭代和发布计划测试时,我们可以借鉴上下文驱动测试学派的重要原则。
We can borrow important principles from the context-driven school of testing when planning testing for each story, iteration, and release.
任何实践的价值都取决于它的背景。
The value of any practice depends on its context.
有上下文中的良好实践,但没有最佳实践。
There are good practices in context, but there are no best practices.
一起工作的人是任何项目环境中最重要的部分。
People, working together, are the most important part of any project’s context.
项目随着时间的推移以通常无法预测的方式展开。
Projects unfold over time in ways that are often not predictable.
该产品是一种解决方案。如果问题没有解决,产品就无法工作。
The product is a solution. If the problem isn’t solved, the product doesn’t work.
好的软件测试是一个具有挑战性的智力过程。
Good software testing is a challenging intellectual process.
只有通过判断力和技巧,在整个项目中协同运用,我们才能在正确的时间做正确的事情,以有效地测试我们的产品。
Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.
有关上下文驱动测试的更多信息,请访问www.context-driven-testing.com。
For more on context-driven testing, see www.context-driven-testing.com.
象限有助于为敏捷测试实践提供背景信息,但您和您的团队必须在进行过程中进行调整。测试人员帮助提供团队调整和更好地工作所需的反馈。使用您的技能在每次迭代和发布中吸引客户。请注意您的团队何时需要超出当前可用范围的角色或知识。
The quadrants help give context to agile testing practices, but you and your team will have to adapt as you go. Testers help provide the feedback the team needs to adjust and work better. Use your skills to engage the customers throughout each iteration and release. Be conscious of when your team needs roles or knowledge beyond what it currently has available.
敏捷测试象限提供了一个清单,以确保您已经覆盖了所有测试基础。检查以下问题的答案:
The Agile Testing Quadrants provide a checklist to make sure you’ve covered all your testing bases. Examine the answers to questions such as these:
我们是否使用单元和组件测试来帮助我们为我们的应用程序找到正确的设计?
Are we using unit and component tests to help us find the right design for our application?
我们是否有一个自动化的构建过程来运行我们的自动化单元测试以获得快速反馈?
Do we have an automated build process that runs our automated unit tests for quick feedback?
我们面向业务的测试是否有助于我们交付符合客户期望的产品?
Do our business-facing tests help us deliver a product that matches customers’ expectations?
我们是否捕获了所需系统行为的正确示例?我们需要更多吗?我们的测试是否基于这些示例?
Are we capturing the right examples of desired system behavior? Do we need more? Are we basing our tests on these examples?
在开始编码之前,我们是否向用户展示 UI 和报告的原型?用户能否将它们与最终软件的工作方式联系起来?
Do we show prototypes of UIs and reports to the users before we start coding them? Can the users relate them to how the finished software will work?
我们是否为探索性测试预留了足够的时间?我们如何处理可用性测试?我们是否让客户参与得足够多?
Do we budget enough time for exploratory testing? How do we tackle usability testing? Are we involving our customers enough?
我们是否在开发周期的早期就考虑了性能和安全性等技术要求?我们是否有合适的工具来进行“能力”测试?
Do we consider technological requirements such as performance and security early enough in the development cycle? Do we have the right tools to do “ility” testing?
使用矩阵作为地图开始。进行试验,并使用回顾来不断改进您的工作,以通过测试指导开发,并在您通过测试了解的产品知识的基础上进行构建。
Use the matrix as a map to get started. Experiment, and use retrospectives to keep improving your efforts to guide development with tests and build on what you learn about your product through testing.
在本章中,我们介绍了敏捷测试象限作为对测试进行分类的便捷方式。四个象限作为指导,以确保产品质量的所有方面都包含在测试和开发过程中。
In this chapter we introduced the Agile Testing Quadrants as a convenient way to categorize tests. The four quadrants serve as guidelines to ensure that all facets of product quality are covered in the testing and developing process.
支持团队的测试可用于推动需求。
Tests that support the team can be used to drive requirements.
评价产品的测试有助于我们思考应用程序质量的各个方面。
Tests that critique the product help us think about all facets of application quality.
使用象限了解您何时完成,并确保整个团队共同负责覆盖矩阵的四个象限。
Use the quadrants to know when you’re done, and ensure the whole team shares responsibility for covering the four quadrants of the matrix.
管理技术债务是任何软件开发团队的重要基础。使用象限来考虑不同的维度。
Managing technical debt is an essential foundation for any software development team. Use the quadrants to think about the different dimensions.
上下文应始终指导我们的测试工作。
Context should always guide our testing efforts.
我们使用敏捷测试象限作为指南来帮助我们涵盖我们需要的所有测试类型,并帮助我们确保我们拥有正确的资源以在每种类型上取得成功。在本章中,我们将研究第一象限中的测试、支持团队的面向技术的测试,以及支持该测试的工具。该象限中的活动构成了敏捷开发的核心。
We use the Agile Testing Quadrants as a guide to help us cover all the types of testing we need and to help us make sure we have the right resources to succeed at each type. In this chapter, we look at tests in the first quadrant, technology-facing tests that support the team, and at tools to support this testing. The activities in this quadrant form the core of agile development.
我们首先讨论象限 1,因为支持团队的面向技术的测试构成了敏捷开发和测试的基础。请参阅图 7-1以提醒敏捷测试象限,该象限已突出显示。第一象限不仅仅是测试。我们在象限 1 中讨论的单元和组件测试并不是为每个故事编写的第一个测试,但它们有助于指导设计和开发。没有测试驱动的基础设计、自动化单元和组件测试,以及运行测试的持续集成过程,很难及时交付价值。其他象限的所有测试都无法弥补这一象限的不足。我们将在接下来的几章中讨论其他象限,并解释它们是如何组合在一起的。
We discuss Quadrant 1 first because the technology-facing tests that support the team form the foundation of agile development and testing. See Figure 7-1 for a reminder of the Agile Testing Quadrants with this quadrant highlighted. Quadrant 1 is about much more than testing. The unit and component tests we talk about in Quadrant 1 aren’t the first tests written for each story, but they help guide design and development. Without a foundation of test-driven design, automated unit and component tests, and a continuous integration process to run the tests, it’s hard to deliver value in a timely manner. All of the testing in the other quadrants can’t make up for inadequacies in this one. We’ll talk about the other quadrants in the next few chapters and explain how they all fit together.
团队需要正确的工具和流程来创建和执行指导开发的面向技术的测试。我们将给出本章最后一节所需工具类型的一些示例。
Teams need the right tools and processes to create and execute technology-facing tests that guide development. We’ll give some examples of the types of tools needed in the last section of this chapter.
单元测试和组件测试通过帮助程序员准确理解代码需要做什么以及通过提供指导来确保质量正确的设计。他们帮助团队专注于正在交付的故事,并采取最简单的方法。单元测试验证小到单个对象或方法的部件的行为 [ Meszaros, 2007 ]。组件测试通过测试类或方法之间的交互来帮助巩固系统可部署部分的整体设计。
Unit tests and component tests ensure quality by helping the programmer understand exactly what the code needs to do, and by providing guidance in the right design. They help the team to focus on the story that’s being delivered and to take the simplest approach that will work. Unit tests verify the behavior of parts as small as a single object or method [Meszaros, 2007]. Component tests help solidify the overall design of a deployable part of the system by testing the interaction between classes or methods.
在使用 TDD 时,开发单元测试可能是必不可少的设计工具。当敏捷程序员开始编码任务时,她会编写一个测试来捕获一小段代码的行为,然后处理代码直到测试通过。通过以小的测试-代码-测试增量构建代码,程序员有机会全面考虑客户需要的功能。出现问题时,她可以询问客户。她可以与测试人员配对,以帮助确保该段代码的所有方面及其与其他单元的通信都经过测试。
Developing unit tests can be an essential design tool when using TDD. When an agile programmer starts a coding task, she writes a test that captures the behavior of a tiny bit of code and then works on the code until the test passes. By building the code in small test-code-test increments, the programmer has a chance to think through the functionality that the customer needs. As questions come up, she can ask the customer. She can pair with a tester to help make sure all aspects of that piece of code, and its communication with other units, are tested.
测试驱动开发这个术语会误导那些不明白它更多的是设计而不是测试的从业者。测试先行开发的代码自然是为可测试性而设计的。象限 1 的活动都旨在生产具有尽可能高的内部质量的软件。
The term test-driven development misleads practitioners who don’t understand that it’s more about design than testing. Code developed test-first is naturally designed for testability. Quadrant 1 activities are all aimed at producing software with the highest possible internal quality.
当团队实践 TDD 时,他们会最大限度地减少以后必须发现的错误数量。通过在代码之前编写测试来防止大多数单元级错误。通过编写单元测试来思考设计意味着系统更有可能满足客户的要求。当后期开发测试时间被用于查找和修复程序员测试可能检测到的错误时,就没有时间查找可能对业务产生不利影响的严重问题。从我们的编码过程中泄漏的错误越多,我们的交付就会越慢,最终,质量会受到影响。这就是象限 1 中的程序员测试如此重要的原因。虽然每个团队都应该采用适合其情况的做法,
When teams practice TDD, they minimize the number of bugs that have to be caught later on. Most unit-level bugs are prevented by writing the test before the code. Thinking through the design by writing the unit test means the system is more likely to meet customer requirements. When post-development testing time is occupied with finding and fixing bugs that could have been detected by programmer tests, there’s no time to find the serious issues that might adversely affect the business. The more bugs that leak out of our coding process, the slower our delivery will be, and in the end, it is the quality that will suffer. That’s why the programmer tests in Quadrant 1 are so critical. While every team should adopt practices that work for its situation, a team without these core agile practices is unlikely to benefit much from agile values and principles.
可靠的源代码控制、配置管理和持续集成对于从指导开发的程序员测试中获取价值至关重要。它们使团队始终能够准确了解正在测试的内容。持续集成为我们提供了一种在每次签入新代码时运行测试的方法。当测试失败时,我们知道是谁签入了导致失败的更改,并且那个人可以快速解决问题。连续的集成节省了时间并激励每个程序员在签入新代码之前运行测试。持续集成和构建过程提供了一个可部署的代码包供我们测试。
Solid source code control, configuration management, and continuous integration are essential to getting value from programmer tests that guide development. They enable the team to always know exactly what’s being tested. Continuous integration gives us a way to run tests every time new code is checked in. When a test fails, we know who checked in the change that caused the failure, and that person can quickly fix the problem. Continuous integration saves time and motivates each programmer to run the tests before checking in the new code. A continuous integration and build process delivers a deployable package of code for us to test.
缺乏这些核心敏捷实践的敏捷项目往往会变成“迷你瀑布”。开发周期更短了,但代码仍然被“翻墙”扔给测试人员,因为代码质量很差,他们没有时间进行测试。瀑布这个词不一定是贬义的。我们从事过成功的“瀑布式”项目,在这些项目中,程序员严格自动化单元测试,实践持续集成,并使用自动化构建来运行测试。这些成功的“瀑布式”项目还涉及整个开发周期的客户和测试人员。当我们在没有适当的实践和工具的情况下进行编码时,无论我们如何称呼这个过程,我们都不会及时交付高质量的代码。
Agile projects that lack these core agile practices tend to turn into “mini-waterfalls.” The development cycles are shorter, but code is still being thrown “over the wall” to testers who run out of time to test because the code is of poor quality. The term waterfall isn’t necessarily derogatory. We’ve worked on successful “waterfall” projects where the programmers stringently automate unit tests, practice continuous integration, and use automated builds to run tests. These successful “waterfall” projects also involve customers and testers throughout the development cycle. When we code without appropriate practices and tools, regardless of what we call the process, we’re not going to deliver high-quality code in a timely manner.
我们不会在这里详细介绍如何进行 TDD,或者编写单元和组件测试的最佳方法。有几本关于这些主题的优秀书籍。我们的目标是解释为什么这些活动对敏捷测试人员很重要。让我们探讨一些使用支持团队的面向技术的测试的原因。
We’re not going into any details here about how to do TDD, or the best ways to write unit and component tests. There are several excellent books on those subjects. Our goal is to explain why these activities are important to agile testers. Let’s explore some reasons to use technology-facing tests that support the team.
速度永远不应该是敏捷开发团队的最终目标。试图快速做事并在紧迫的期限内完成任务而不考虑质量会导致我们偷工减料并重拾旧有的坏习惯。如果我们偷工减料,我们就会积累更多的技术债务,并且可能无论如何都会错过最后期限。令人高兴的是,速度是生成具有最高内部质量的代码的长期副作用。持续构建运行单元测试会在问题签入后的几分钟内通知团队失败,并且可以快速找到并修复错误。自动化单元和代码集成测试的安全网使程序员能够经常重构。这使代码保持在合理的可维护性标准,并为投入的时间提供最佳价值。技术债务保持在尽可能低的水平。
Speed should never be the end goal of an agile development team. Trying to do things fast and meet tight deadlines without thinking about the quality causes us to cut corners and revert to old, bad habits. If we cut corners, we’ll build up more technical debt, and probably miss the deadline anyway. Happily, though, speed is a long-term side effect of producing code with the highest possible internal quality. Continuous builds running unit tests notify the team of failure within a few minutes of the problem check-in, and the mistake can be found and fixed quickly. A safety net of automated unit and code integration tests enables the programmers to refactor frequently. This keeps the code at a reasonable standard of maintainability and delivers the best value for the time invested. Technical debt is kept as low as possible.
如果您在一个忽略了单元测试的项目中担任过测试人员,您就会知道将所有时间都花在查找单元级缺陷上是多么容易。在测试“快乐路径”时,您可能会发现如此多的错误,以至于您永远没有时间测试更复杂的场景和边缘情况。随着“查找和修复”周期的拖延,发布截止日期被推迟,或者测试刚刚停止并且有缺陷的产品被强加给毫无戒心的客户。
If you’ve worked as a tester on a project where unit testing was neglected, you know how easy it is to spend all of your time finding unit-level defects. You might find so many bugs while testing the “happy path” that you never have time to test more complex scenarios and edge cases. The release deadline is pushed back as the “find and fix” cycle drags on, or testing is just stopped and a buggy product is foisted off on unsuspecting customers.
与这种情况相比,我们在敏捷团队中的岁月一直是乌托邦。通过测试推动编码实践意味着程序员可能相当了解故事的要求。他们与客户和测试人员进行了广泛的交谈,以阐明所需的行为。各方都了解正在进行的更改。当团队完成编写一个故事的所有任务卡,或一个薄的、可测试的片段时,单元和组件测试已经很好地覆盖了该功能。通常,程序员已经确保故事中至少有一条路径可以从头到尾运行。
Our years on agile teams have been Utopian in contrast to this scenario. Driving coding practices with tests means that the programmers probably understood the story’s requirements reasonably well. They’ve talked extensively with the customers and testers to clarify the desired behaviors. All parties understand the changes being made. By the time the team has completed all of the task cards for coding a story, or a thin, testable slice of one, the feature has been well covered by unit and component tests. Usually the programmers have made sure at least one path through the story works end to end.
这意味着我们作为测试人员几乎不会浪费时间寻找低级错误。我们可能会尝试程序员没有想到的场景,并将时间花在更高级别的业务功能上。设计良好的代码通常是健壮的和可测试的。如果我们发现缺陷,我们会把它展示给程序员,程序员编写单元测试来重现错误,然后快速修复它。实际上,我们有时间专注于探索性测试和其他类型的深入测试,以便对代码进行良好的锻炼,并了解更多关于它应该如何工作的信息。通常,我们发现的唯一“错误”是我们团队中的每个人都错过或误解的要求。如果客户参与并有定期的演示和测试机会,即使是那些也能很快找到。开发团队掌握 TDD 后,
This means that we, as testers, waste little time finding low-level bugs. We’re likely to try scenarios the programmers hadn’t thought of and to spend our time on higher-level business functionality. Well-designed code is usually robust and testable. If we find a defect, we show it to the programmer, who writes a unit test to reproduce the bug and then fixes it quickly. We actually have time to focus on exploratory testing and the other types of in-depth tests to give the code a good workout and learn more about how it should work. Often, the only “bugs” we find are requirements that everyone on our team missed or misunderstood. Even those are found quickly if the customer is involved and has regular demos and test opportunities. After a development team has mastered TDD, the focus for improvement shifts from bug prevention to figuring out better ways to elicit and capture requirements before coding.
关于何时编写测试以及出于什么目的,存在多种不同的理念。尽管敏捷社区普遍认为 TDD 确实可以帮助团队实现质量更好的软件,但每个团队都应就有助于其实现质量目标的方法达成一致。这是程序员测试支持团队的重要方式。让我们看看更多的方法。
There are several different philosophies about when to write tests and for what purpose. It’s up to each team to agree on the approach that helps it achieve its quality objectives, although there is common agreement in the agile community that TDD definitely helps a team achieve better-quality software. This is an important way that programmer tests support the team. Let’s look at some more ways.
与程序员测试相关的核心实践使许多测试活动更容易完成。程序员在他们自己的沙盒中工作,他们可以在其中测试新代码而不影响其他人的工作。他们不会签入代码,直到代码在他们的沙箱中通过了一套回归测试。
The core practices related to programmer tests make lots of testing activities easier to accomplish. Programmers work in their own sandboxes, where they can test new code without affecting anyone else’s work. They don’t check in code until it has passed a suite of regression tests in their sandbox.
团队考虑测试环境以及测试数据使用什么。单元测试通常使用假对象或模拟对象而不是实际数据库来提高速度,但程序员仍然需要针对真实数据进行测试。测试人员可以帮助他们识别好的测试数据。如果单元测试代表现实生活中的数据,以后发现的问题就会更少。
The team thinks about test environments and what to use for test data. Unit tests usually work with fake or mock objects instead of actual databases for speed, but programmers still need to test against realistic data. Testers can help them identify good test data. If the unit tests represent real-life data, fewer issues will be found later.
编写测试和编写代码时考虑到这些测试意味着程序员总是有意识地使代码可测试。所有这些与基础架构相关的良好品质都会溢出到面向业务的测试和批评产品的测试中。整个团队一直在思考改进设计和简化测试的方法。
Writing tests and writing code with those tests in mind means programmers are always consciously making code testable. All of these good infrastructure-related qualities spill over to business-facing tests and tests that critique the product. The whole team is continually thinking of ways to improve design and make testing easier.
用测试驱动开发的优势之一是编写代码的目的是让测试通过。团队必须从一开始就考虑如何为其编码的每个故事执行和自动化测试。测试驱动开发意味着程序员在编写代码之前会编写每个测试以使其通过。
One advantage of driving development with tests is that code is written with the express intention of making the tests pass. The team has to think, right from the beginning, about how it will execute and automate tests for every story it codes. Test-driven development means that programmers will write each test before they write the code to make it pass.
编写“可测试的代码”是一个简单的概念,但这并不是一件容易的事,尤其是当您正在处理没有自动测试且不是为可测试性而设计的旧代码时。遗留系统通常将业务逻辑、I/O、数据库和用户界面层交织在一起。没有简单的方法来挂钩以在 GUI 下或单元级别自动化测试。
Writing “testable code” is a simple concept, but it’s not an easy task, especially if you’re working on old code that has no automated tests and isn’t designed for testability. Legacy systems often have business logic, I/O, database, and user interface layers intertwined. There’s no easy way to hook in to automate a test below the GUI or at the unit level.
设计可测试架构的常用方法是将应用程序中执行不同功能的不同层分开。理想情况下,您希望使用测试夹具和具有不同输入的测试算法直接访问每一层。为此,您将业务逻辑隔离到它自己的层中,使用假对象而不是尝试访问其他应用程序或实际数据库。如果可以将表示层与底层业务逻辑和数据库访问分离,则可以在不测试底层逻辑的情况下快速测试输入验证。
A common approach in designing a testable architecture is to separate the different layers that perform different functions in the application. Ideally, you would want to access each layer directly with a test fixture and test algorithms with different inputs. To do this, you isolate the business logic into its own layer, using fake objects instead of trying to access other applications or the actual database. If the presentation layer can be separated from underlying business logic and database access, you can quickly test input validation without testing underlying logic.
可测试设计方法的另一个例子是 Alistair Cockburn 的端口和适配器模式 [Cockburn, 2005]。此模式的目的是“创建无需 UI 或数据库即可工作的应用程序,以便您可以针对应用程序运行自动回归测试,在数据库不可用时工作,并将应用程序链接在一起而无需任何用户参与。” 端口接受外部事件,特定技术的适配器将其转换为应用程序可以理解的消息。反过来,应用程序通过端口将输出发送到适配器,适配器创建接收人员或自动用户所需的信号。使用此模式设计的应用程序可以像实际用户一样轻松地由自动化测试脚本驱动。
Another example of an approach to testable design is Alistair Cockburn’s Ports and Adapters pattern [Cockburn, 2005]. This pattern’s intent is to “create your application to work without either a UI or a database so you can run automated regression tests against the application, work when the database becomes unavailable, and link applications together without any user involvement.” Ports accept outside events, and a technology-specific adapter converts it into a message that can be understood by the application. In turn, the application sends output via a port to an adapter, which creates the signals needed by the receiving human or automated users. Applications designed using this pattern can be driven by automated test scripts as easily as by actual users.
有关 Alastair Cockburn 的端口和适配器模式的更多信息,请参阅参考书目。
See the bibliography for more information on Alastair Cockburn’s Ports and Adapters pattern.
更明显的是如何在未开发项目上编写测试先行的代码。自动化单元测试未涵盖的遗留系统提出了巨大的挑战。很难为不是为可测试性而设计的代码编写单元测试,也很难更改不受单元测试保护的代码。很多球队都有遵循 Michael Feathers 在Working Effectively with Legacy Code [Feathers, 2004]中解释的“遗留代码救援”技术。其他团队,例如 Lisa 的团队,旨在“扼杀”他们的遗留代码。该策略源于 Martin Fowler 的“扼杀器应用程序”[ Fowler, 2004 ]。在仍然维护旧系统的同时,新故事在新架构中以测试优先的方式编码。随着时间的推移,系统的大部分已转换为新架构,目标是最终废除旧系统。
It’s more obvious how to code test-first on a greenfield project. Legacy systems, which aren’t covered by automated unit tests, present a huge challenge. It’s hard to write unit tests for code that isn’t designed for testability, and it’s hard to change code that isn’t safeguarded with unit tests. Many teams have followed the “legacy code rescue” techniques explained by Michael Feathers in Working Effectively with Legacy Code [Feathers, 2004]. Other teams, such as Lisa’s, aim to “strangle” their legacy code. This strategy stems from Martin Fowler’s “strangler application” [Fowler, 2004]. New stories were coded test-first in a new architecture while the old system was still maintained. Over time, much of the system has been converted to the new architecture, with the goal of eventually doing away with the old system.
参考书目有更多关于“拯救”和“扼杀”遗留代码方法的文章的链接。
The bibliography has links to more articles about “rescue” and “strangler” approaches to legacy code.
在遗留大型机类型的环境中进行敏捷测试会带来特殊的挑战,其中最重要的是缺乏有关如何成功进行测试的出版物和信息。COBOL、大型机及其同类产品仍在广泛使用。当您寻找在应用程序中启用自动化测试的方法时,让敏捷原则和价值观指导您的团队。您可能需要调整一些技巧;例如,也许你不能写代码测试先行,但你可以在写完代码后马上进行测试。当需要解决的是团队的问题,而不仅仅是测试人员的问题时,您就会找到编写测试的方法。
Agile testing in a legacy mainframe type of environment presents particular challenges, not the least of which is the lack of availability of publications and information about how to do it successfully. COBOL, mainframes, and their ilk are still widely used. Let agile principles and values guide your team as you look for ways to enable automated testing in your application. You might have to adapt some techniques; for example, maybe you can’t write code test-first, but you can test soon after writing the code. When it’s the team’s problem to solve, and not just the testers’ problem, you’ll find a way to write tests.
有关 RPGUnit 的更多信息,请参阅www.RPGUnit.org。
For more about RPGUnit, see www.RPGUnit.org.
您的团队可以找到适合您的可测试性设计方法。秘诀在于整个团队对测试和质量的承诺。当一个团队不断努力编写测试并使它们通过时,它就会找到完成它的方法。团队应该花时间考虑他们如何创建一个架构,使自动化测试易于创建、维护成本低且寿命长。如果自动化测试没有为投资回报足够的价值,不要害怕重新审视架构。
Your team can find an approach to designing for testability that works for you. The secret is the whole-team commitment to testing and quality. When a team is constantly working to write tests and make them pass, it finds a way to get it done. Teams should take time to consider how they can create an architecture that will make automated tests easy to create, inexpensive to maintain, and long-lived. Don’t be afraid to revisit the architecture if automated tests don’t return enough value for the investment in them.
单元测试的最大价值在于其反馈的速度。我们认为,运行单元测试的持续集成和构建过程应该在十分钟内完成。如果每个程序员每天检查代码几次,那么较长的构建和测试过程将导致更改开始堆积。作为测试人员,不得不长时间等待新功能或错误修复可能会令人沮丧。如果出现编译错误或单元测试失败,延迟会变得更糟,尤其是快到回家的时候了!
The biggest value of unit tests is in the speed of their feedback. In our opinion, a continuous integration and build process that runs the unit tests should finish within ten minutes. If each programmer checks code in several times a day, a longer build and test process will cause changes to start stacking up. As a tester, it can be frustrating to have to wait a long time for new functionality or a bug fix. If there’s a compile error or unit test failure, the delay gets even worse, especially if it’s almost time to go home!
在单元级别之上运行测试的构建和测试过程(例如功能性 API 测试或 GUI 测试)将花费更长的时间。至少有一个快速运行的构建过程,第二个运行较慢的测试。应该至少有一个每日“构建”来运行所有较慢的功能测试。然而,即使那样也可能很笨拙。当测试失败并且问题得到解决时,需要多长时间才能确定构建再次通过?
A build and test process that runs tests above the unit level, such as functional API tests or GUI tests, is going to take longer. Have at least one build process that runs quickly, and a second that runs the slower tests. There should be at least one daily “build” that runs all of the slower functional tests. However, even that can be unwieldy. When a test fails and the problem is fixed, how long will it take to know for sure that the build passes again?
如果您的构建和测试过程花费的时间太长,请让您的团队分析减速的原因并采取措施加快构建速度。这里有一些例子。
If your build and test process takes too long, ask your team to analyze the cause of the slowdown and take steps to speed up the build. Here are a few examples.
数据库访问通常会消耗大量时间,因此请考虑在可能的情况下使用假对象来替换数据库,尤其是在单元级别。
Database access usually consumes lots of time, so consider using fake objects, where possible, to replace the database, especially at the unit level.
将运行时间较长的集成和数据库访问测试移至辅助构建和测试过程。
Move longer-running integration and database-access tests to the secondary build and test process.
查看测试是否可以并行运行,以便它们更快地完成。
See if tests can run in parallel so that they finish faster.
Run the minimum tests needed for regression testing your system.
在多台构建机器上分配任务。
Distribute tasks across multiple build machines.
升级运行构建的硬件和软件。
Upgrade the hardware and software that run the build.
找到花费最多时间的区域并采取渐进的步骤来加快速度。
Find the area that takes the most time and take incremental steps to speed it up.
哇,多个连续的构建和测试过程提供持续的反馈——这对很多测试人员来说听起来像是一个梦想。回归错误将在修复成本最低的时候及早发现。这是编写面向技术的测试的重要原因。但是,我们会不会对他们太着迷了?让我们看看面向技术的测试和面向业务的测试之间的界限。
Wow, multiple continuous build and test processes providing constant feedback—it sounds like a dream to a lot of testers. Regression bugs will be caught early, when they’re cheapest to fix. This is a great reason for writing technology-facing tests. Can we get too carried away with them, though? Let’s look at the line between technology-facing tests and business-facing tests.
我们经常听到人们担心面向客户的测试会与面向技术的测试重叠太多,以至于团队会浪费时间。我们知道面向业务的测试可能涵盖与单元或代码集成测试相同的领域,但它们的目的如此不同,因此不必担心浪费。
We often hear people worry that the customer-facing tests will overlap so much with the technology-facing tests that the team will waste time. We know that business-facing tests might cover a bit of the same ground as unit or code integration tests, but they have such different purposes that waste isn’t a worry.
例如,我们有一个故事来计算贷款分期偿还时间表并将其显示给正在申请贷款的用户。这个故事的单元测试可能会测试非法参数,例如,如果业务不允许,则每年支付频率。可能有一个单元测试来确定预期的贷款支付开始日期,给定金额、利率、开始日期和频率的一些定义。单元级测试可以涵盖支付频率、金额、利息日期、期限和开始日期的不同组合,以证明摊销计算是正确的。它们可以涵盖闰年等情景。当这些测试通过时,程序员会对代码充满信心。
For example, we have a story to calculate a loan amortization schedule and display it to a user who’s in the process of requesting a loan. A unit test for this story would likely test for illegal arguments, such as an annual payment frequency if the business doesn’t allow it. There might be a unit test to figure the anticipated loan payment start date given some definition of amount, interest rate, start date, and frequency. Unit-level tests could cover different combinations of payment frequency, amount, interest date, term, and start date in order to prove that the amortization calculation is correct. They could cover scenarios such as leap years. When these tests pass, the programmer feels confident about the code.
每个单元测试都是独立的,一次测试一个维度。这意味着当单元测试失败时,程序员可以快速识别问题并尽快解决问题。面向业务的测试很少只涵盖一个维度,因为它们是从业务角度处理的。
Each unit test is independent and tests one dimension at a time. This means that when a unit test fails, the programmer can identify the problem quickly and solve the issue just as quickly. The business-facing tests very seldom cover only one dimension, because they are tackled from a business point of view.
这个故事的面向业务的测试将为业务规则、用户界面中的表示和错误处理定义更多细节。他们将验证支付细节,例如应用的本金和利息,是否正确显示在用户界面中。他们将测试用户界面上每个字段的验证,并指定针对余额不足或不合格等情况的错误处理。他们可以测试管理员在同一天处理两笔贷款支付的场景,这可能更难在单位级别模拟。
The business-facing tests for this story would define more details for the business rules, the presentation in the user interface, and error handling. They would verify that payment details, such as the principal and interest applied, display correctly in the user interface. They would test validations for each field on the user interface, and specify error handling for situations such as insufficient balance or ineligibility. They could test a scenario where an administrator processes two loan payments on the same day, which might be harder to simulate at the unit level.
面向业务的测试涵盖更复杂的用户场景,验证最终用户是否有良好的体验。尽可能将测试推到较低的级别;如果您确定了一个可以在单元级别自动化的测试用例,那几乎总能获得更好的投资回报。
The business-facing tests cover more complex user scenarios and verify that the end user will have a good experience. Push tests to lower levels whenever possible; if you identify a test case that can be automated at the unit level, that’s almost always a better return on investment.
第 13 章,“为什么我们想要自动化测试以及是什么阻碍了我们”,更多地讨论了不同类型测试的投资回报率。
Chapter 13, “Why We Want to Automate Tests and What Holds Us Back,” talks more about the ROI of the different types of tests.
如果涉及应用程序的多个区域或层,则可能无法在单元级别实现自动化。面向技术和面向业务的级别都可能在第一次贷款支付日期前后进行测试,但它们出于不同的原因进行测试。单元测试将检查日期的计算,面向业务的测试将验证它是否正确显示在借款人的贷款报告中。
If multiple areas or layers of the application are involved, it might not be possible to automate at the unit level. Both technology-facing and business-facing levels might have tests around the date of the first loan payment, but they check for different reasons. The unit test would check the calculation of the date, and the business-facing test would verify that it displays correctly in the borrower’s loan report.
学习编写象限 1 测试很难。许多过渡到敏捷开发的团队开始时没有自动化单元测试,甚至没有持续集成和构建过程。在下一节中,我们建议敏捷测试人员在他们的团队不处理象限 1 测试时可以采取的行动。
Learning to write Quadrant 1 tests is hard. Many teams making the transition to agile development start out with no automated unit tests, not even a continuous integration and build process. In the next section, we suggest actions agile testers can take if their teams don’t tackle Quadrant 1 tests.
许多组织已经决定尝试敏捷开发,或者至少表明了这种意图,但并不了解如何进行成功的转换。当我们担任测试人员角色时,我们可以做些什么来帮助开发团队实施 TDD、持续集成和其他成功开发的关键实践?
Many an organization has decided to try agile development, or at least stated that intention, without understanding how to make a successful transition. When we’re in a tester role, what can we do to help the development team implement TDD, continuous integration, and other practices that are key to successful development?
多年来我们的经验是,如果我们自己不是程序员,那么在敦促程序员采用 TDD 等实践时,我们不一定有多少可信度。如果我们可以坐下来向他们展示如何编写测试先行的代码,那将是有说服力的,但我们中的许多测试人员都没有那种经验。我们还发现传福音不起作用。从概念上来说服某人 TDD 是个好主意并不难。帮助他们获得牵引力实际编码测试先行要困难得多。
Our experience over the years has been that if we aren’t programmers ourselves, we don’t necessarily have much credibility when we urge the programmers to adopt practices such as TDD. If we could sit down and show them how to code test-first, that would be persuasive, but many of us testers don’t have that kind of experience. We’ve also found that evangelizing doesn’t work. It’s not that hard to convince someone conceptually that TDD is a good idea. It’s much trickier to help them get traction actually coding test-first.
如果您是所谓的“敏捷”团队中的测试人员,该团队甚至不执行单元测试自动化或生成连续构建(或者至少不每天进行构建),您很快就会感到沮丧。不要放弃;继续集思广益,寻找一种方法来推动积极的转变。尝试利用社交时间或其他放松活动来花一些时间,看看您可以产生哪些新想法来让所有团队成员参与进来。
If you’re a tester on a so-called “agile” team that isn’t even automating unit tests or producing continuous builds—or at a minimum, doing builds on a daily basis—you’re going to get frustrated pretty quickly. Don’t give up; keep brainstorming for a way to get traction on a positive transition. Try using social time or other relaxing activity to take some quality time to see what new ideas you can generate to get all team members on board.
要避免的一个陷阱是让测试人员编写单元测试。因为 TDD 实际上更像是一种设计活动,所以编写代码的人在编写代码之前也必须编写测试。程序员还需要自动化单元测试提供的即时反馈。在编写代码后由其他人编写的单元测试可能仍然可以防止回归缺陷,但它们不会获得程序员编写的测试的最有价值的好处。
One trap to avoid is having testers write the unit tests. Because TDD is reallymore of a design activity, it’s essential that the person writing the code also write the tests, before writing the code. Programmers also need the immediate feedback that automated unit tests give. Unit tests written by someone else after the code is written might still guard against regression defects, but they won’t have the most valuable benefits of tests written by the programmer.
作为敏捷团队的测试人员,您可以做很多事情来充当变更推动者,但您的潜在影响是有限的。在某些情况下,强大的管理支持是推动团队从事第一象限活动的关键。
As a tester on an agile team, there’s a lot you can do to act as a change agent, but your potential impact is limited. In some cases, strong management support is the key to driving the team to engage in Quadrant 1 activities.
如果您正在管理一个开发团队,您可以做很多事情来鼓励测试驱动开发和单元测试自动化。与产品负责人合作,将质量作为您的目标,并将质量标准传达给团队。鼓励程序员花时间把工作做到最好,而不是担心赶不上最后期限。如果交货日期处于危险之中,则推动缩小范围,而不是质量。你的工作是向业务经理解释将质量放在首位将如何确保他们获得最佳业务价值。
If you’re managing a development team, you can do a lot to encourage test-driven development and unit test automation. Work with the product owner to make quality your goal, and communicate the quality criteria to the team. Encourage the programmers to take time to do their best work instead of worrying about meeting a deadline. If a delivery date is in jeopardy, push to reduce the scope, not the quality. Your job is to explain to the business managers how making quality a priority will ensure that they get optimum business value.
给团队时间学习,并提供专业的实践培训。引入经验丰富的敏捷开发教练或聘请具有使用这些实践经验的人,他们可以将这些技能传授给团队的其他成员。为重大重构、就编写单元和代码集成测试的最佳方法集思广益以及评估、安装和升级工具安排时间。测试经理应该与开发一起工作管理人员鼓励增强可测试性并允许测试人员编写可执行测试的实践。测试经理还可以确保测试人员有时间学习如何使用团队决定实施的自动化工具和框架。
Give the team time to learn, and provide expert, hands-on training. Bring in an experienced agile development coach or hire someone with experience in using these practices who can transfer those skills to the rest of the team. Budget time for major refactoring, for brainstorming about the best approach to writing unit and code integration tests, and for evaluating, installing, and upgrading tools. Test managers should work with development managers to encourage practices that enhance testability and allow testers to write executable tests. Test managers can also make sure testers have time to learn how to use the automation tools and frameworks that the team decides to implement.
虽然您可以找到成为有效变革推动者的方法,但最好的办法是让整个团队参与解决问题。如果您还没有在每次迭代后进行回顾,建议尝试这种做法或其他类型的流程改进。在回顾中,提出阻碍成功交付的问题。例如,“我们没有在迭代结束前完成测试任务”是整个团队要解决的问题。如果未完成的原因之一是大量的单元级错误,建议尝试使用 TDD,但允许程序员提出他们自己的解决问题的方法。鼓励团队尝试一种新方法进行几次迭代,看看它是如何工作的。
While you can find ways to be an effective change agent, the best thing to do is involve the whole team in solving the problems. If you aren’t already doing retrospectives after every iteration, propose trying this practice or some other type of process improvement. At the retrospective, raise issues that are hampering successful delivery. For example, “We aren’t finishing testing tasks before the end of the iteration” is a problem for the whole team to address. If one reason for not finishing is the high number of unit-level bugs, suggest experimenting with TDD, but allow programmers to propose their own ways to address the problem. Encourage the team to try a new approach for a few iterations and see how it works.
有关回顾和过程改进的更多信息,请参阅第 19 章“结束迭代”。
More about retrospectives and process improvement in Chapter 19, “Wrap Up the Iteration.”
支持团队开发过程的面向技术的测试是所有需要进行的测试的重要基础。如果团队在这个象限中的测试做得不够好,那么其他类型的测试就会困难得多。这并不意味着你不能从其他象限中单独获得价值——这只是意味着这样做会更难,因为团队的代码将缺乏内部质量,一切都将花费更长的时间。
Technology-facing tests that support the team’s development process are an important foundation for all of the testing that needs to happen. If the team isn’t doing an adequate job with the tests in this quadrant, the other types of testing will be much more difficult. This doesn’t mean you can’t get value from the other quadrants on their own—it just means it will be harder to do so because the team’s code will lack internal quality and everything will take longer.
没有正确的工具和基础设施,就无法完成面向技术的测试。在下一节中,我们将查看团队需要有效进行象限 1 测试的工具类型的示例。
Technology-facing tests can’t be done without the right tools and infrastructure. In the next section, we look at examples of the types of tools a team needs to be effective with Quadrant 1 tests.
没有什么神奇的工具可以确保成功。然而,工具可以帮助优秀的人把工作做到最好。建立正确的基础架构以支持面向技术的测试至关重要。有大量优秀的工具可供选择,而且它们一直在改进。您的团队必须找到最适合您情况的工具。
There’s no magical tool that will ensure success. However, tools can help good people do their best work. Building up the right infrastructure to support technology-facing tests is critical. There’s a huge selection of excellent tools available, and they improve all the time. Your team must find the tools that work best for your situation.
源代码控制也有其他名称,例如版本控制或修订控制。它当然不是新的,也不是敏捷开发所独有的,但是没有它,任何软件开发团队都无法成功。这就是我们在这里讨论它的原因。如果没有源代码控制,您将永远无法确定您正在测试的是什么。程序员是只更改了他说他更改的模块,还是忘记了他对其他模块所做的更改?如果没有某种版本控制系统,您将无法撤销不需要的或错误的更改。源代码控制可以防止不同的程序员在同一模块上互相修改。如果没有版本控制,您将无法确定将哪些代码发布到生产环境中。
Source code control is known by other names too, such as version control or revision control. It’s certainly not new, or unique to agile development, but no software development team can succeed without it. That’s why we’re discussing it here. Without source code control, you’ll never be sure what you’re testing. Did the programmer change only the module he said he changed, or did he forget changes he made to other modules? You can’t back out unwanted or erroneous changes without some kind of versioning system. Source code control keeps different programmers from walking on each other’s changes to the same modules. Without versioning, you can’t be sure what code to release to production.
Software Configuration Management Patterns: Effective Teamwork, Practical Integrations [ 2003 ],作者 Stephen Berczuk 和 Brad Appleton,是学习如何以及为什么使用源代码控制的好资源。源代码控制对于任何风格的软件开发都是必不可少的。
Software Configuration Management Patterns: Effective Teamwork, Practical Integrations [2003], by Stephen Berczuk and Brad Appleton, is a good resource to use to learn how and why to use source code control. Source code control is essential to any style of software development.
也对自动化测试脚本使用源代码控制。将自动化测试与他们测试的相应代码版本联系起来很重要,以防您将来需要针对该版本重新运行测试。当您标记或标记构建时,请确保您也标记或标记测试代码,即使它没有发布到生产环境。
Use source code control for automated test scripts, too. It’s important to tie the automated tests with the corresponding code version that they tested in case you need to rerun tests against that version in the future. When you label or tag a build, make sure you label or tag the test code too, even if it doesn’t get released to production.
团队可以组织他们的代码层次结构,为生产代码、相应的单元测试和更高级别的测试脚本提供存储库。这样做可能需要一些头脑风暴和试验才能获得正确的结构。
Teams can organize their code hierarchy to provide a repository for production code, corresponding unit tests, and higher-level test scripts. Doing this might require some brainstorming and experimenting in order to get the right structure.
有许多很棒的选项可供选择。CVS 和 Subversion (SVN) 等开源系统易于实施,与持续构建过程和 IDE 集成,并且非常健壮。IBM Rational ClearCase 和 Perforce 等供应商工具可能会添加一些功能来补偿它们经常带来的增加的开销。
There are many terrific options to choose from. Open source systems such as CVS and Subversion (SVN) are easy to implement, integrate with a continuous build process and IDEs, and are robust. Vendor tools such as IBM Rational ClearCase and Perforce might add features that compensate for the increased overhead they often bring.
源代码控制与开发环境紧密集成。让我们看一下敏捷团队使用的一些 IDE。
Source code control is tightly integrated with development environments. Let’s look at some IDEs used by agile teams.
一个好的IDE(集成开发环境)可以帮助敏捷团队中的程序员和测试人员。IDE 与源代码控制系统集成,有助于防止版本控制问题和更改相互影响。IDE 内的编辑器特定于编程语言,即使在您编写代码时也会标记错误。最重要的是,IDE 提供了对重构的支持。
A good IDE (integrated development environment) can be helpful for programmers and testers on an agile team. The IDE integrates with the source code control system to help prevent problems with versioning and changes walking on each other. The editors inside an IDE are specific to the programming language and flag errors even as you write the code. Most importantly, IDEs provide support for refactoring.
使用 IDE 的程序员往往具有强烈的个人偏好。但是,有时组织会规定所有程序员都必须使用特定的 IDE。这可能是因为许可,也可能是为了鼓励开放结对编程。如果另一个程序员使用相同的 IDE,则与另一个程序员配对会更容易,但通常不必使用相同的 IDE。大多数工具的工作原理相似,因此不难从一种 IDE 更改为另一种 IDE 以满足新需求或利用新功能。一些顽固分子仍然更喜欢使用经过验证的技术,例如 vi、vim 或带有 make 文件的 emacs 而不是 IDE。
Programmers who use an IDE tend to have strong personal preferences. However, sometimes an organization decrees that all programmers must use a specific IDE. This might be because of licensing, or it might be intended to encourage open pair programming. It is easier to pair with another programmer if the other person uses the same IDE, but it’s generally not essential for the same one to be used. Most tools work similarly, so it’s not hard to change from one IDE to another in order to meet new needs or take advantage of new features. Some diehards still prefer to use tried-and-true technology such as vi, vim, or emacs with make files rather than an IDE.
Eclipse 和 NetBeans 等开源 IDE 以及 Visual Studio 和 IntelliJ IDEA 等专有系统被敏捷团队广泛使用。IDE 具有支持不同语言和工具的插件。它们在测试脚本中的表现与在生产代码中的表现一样好。
Open source IDEs such as Eclipse and NetBeans are widely used by agile teams, along with proprietary systems such as Visual Studio and IntelliJ IDEA. IDEs have plug-ins to support different languages and tools. They work as well with test scripts as they do with production code.
不通过 IDE 进行自动化测试但希望能够查看更改的代码片段的测试人员可以使用 FishEye 等工具,使测试人员能够通过自动构建访问代码。
Testers who aren’t automating tests through an IDE, but who want to be able to look at changed snippets of code, can use tools such as FishEye that enable the testers to get access to the code through the automated build.
在撰写本文时,IDE 已经添加了对动态语言(例如 Ruby、Groovy 和 Python)的支持。使用动态语言的程序员可能更喜欢轻量级的工具,但他们仍然需要支持良好编码实践的好工具,例如 TDD 和重构。
As of this writing, IDEs have added support for dynamic languages such as Ruby, Groovy, and Python. Programmers who use dynamic languages may prefer lighter-weight tools, but they still need good tools that support good coding practices, such as TDD and refactoring.
无论使用何种开发环境和工具,敏捷团队都需要一个框架来集成来自不同程序员的代码更改,运行单元测试以验证没有发生回归错误,并以可部署的格式提供代码。
Regardless of the development environment and tools being used, agile teams need a framework that will integrate code changes from different programmers, run the unit tests to verify no regression bugs have occurred, and provide the code in a deployable format.
您的团队需要一些方法来构建软件并创建可部署的 jar、war 或其他类型的文件。这可以使用基于 shell 的工具(例如 make)来完成,但是这些工具有局限性,例如它们工作的平台。我们知道的敏捷团队使用 ant、Nant 和 Maven 等工具来构建他们的项目。这些工具不仅可以管理构建,还可以提供报告和记录构建结果的简便方法,并且可以轻松地与构建自动化和测试工具集成。它们还与 IDE 集成。
Your team needs some way to build the software and create a deployable jar, war, or other type of file. This can be done with shell-based tools such as make, but those tools have limitations, such as the platforms where they work. Agile teams that we know use tools such as ant, Nant, and Maven to build their projects. These tools not only manage the build but also provide easy ways to report and document build results, and they integrate easily with build automation and test tools. They also integrate with IDEs.
持续集成是敏捷团队的核心实践。您需要一种方法,不仅可以构建项目,还可以对每个构建运行自动化测试,以确保没有任何问题。每天运行多次的完全自动化且可重现的构建是敏捷团队成功的关键因素。自动化构建工具提供构建结果的电子邮件通知等功能,并且它们与构建和源代码控制工具集成。
Continuous integration is a core practice for agile teams. You need a way to not only build the project but also run automated tests on each build to make sure nothing broke. A fully automated and reproducible build that runs many times a day is a key success factor for agile teams. Automated build tools provide features such as email notification of build results, and they integrate with build and source code control tools.
在撰写本书时常用的工具包括开源工具 CruiseControl、CruiseControl.net、CruiseControl.rb 和 Hudson。发布时可用的其他开源和专有工具包括 AnthillPro、Bamboo、BuildBeat、CI Factory、Team City 和 Pulse,仅举几例。
Commonly used tools as of the writing of this book include the open source tools CruiseControl, CruiseControl.net, CruiseControl.rb, and Hudson. Other open source and proprietary tools available at publication time are AnthillPro, Bamboo, BuildBeat, CI Factory, Team City, and Pulse, just to name a few.
如果没有自动构建过程,您将很难部署用于测试和发布的代码。构建管理和构建自动化工具易于实施,并且对于成功的敏捷项目绝对必要。确保尽早开始构建过程,甚至在开始编码之前。当您发现您需要的功能比当前流程提供的更多时,请尝试使用不同的工具。
Without an automated build process you’ll have a hard time deploying code for testing as well as releasing. Build management and build automation tools are easy to implement and absolutely necessary for successful agile projects. Make sure you get your build process going early, even before you start coding. Experiment with different tools when you find you need more features than your current process provides.
单元测试工具特定于您编写代码所用的语言。“xUnit”工具通常被敏捷团队使用,并且有许多不同语言的风格,包括用于 Java 的 JUnit、用于 .NET 的 NUnit、用于 Perl 和 Ruby 的 Test::Unit 以及用于 Python 的 PyUnit。
Unit test tools are specific to the language in which you’re coding. “xUnit” tools are commonly used by agile teams, and there’s a flavor for many different languages, including JUnit for Java, NUnit for .NET, Test::Unit for Perl and Ruby, and PyUnit for Python.
行为驱动开发是测试驱动开发的另一种形式,使用 RSpec 和 easyb 等工具阐明预期行为以驱动测试。
Behavior-driven development is another flavor of test-driven development, spelling out expected behavior to drive tests with tools such as RSpec and easyb.
有关行为驱动开发工具的更多信息,请参阅第 9 章“支持团队的面向业务的测试工具包”。
See Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” for more information on behavior-driven development tools.
GUI 代码也可以而且应该以测试先行的方式开发。用于富客户端单元测试的一些工具是 TestNG、Abbot 和 SWTBot。
GUI code can and should be developed test-first as well. Some tools for rich-client unit testing are TestNG, Abbot, and SWTBot.
EasyMock 和 Ruby/Mock 等工具有助于实现模拟对象和测试存根,这是精心设计的单元测试的组成部分。
Tools such as EasyMock and Ruby/Mock help with implementing mock objects and test stubs, an integral part of well-designed unit tests.
请参阅参考书目中的链接和书籍,以帮助您的团队搜索正确的单元测试工具。
See the bibliography for links and books to help your team search for the right unit test tools.
程序员用来编写面向技术的测试的工具也可以用于面向业务的测试。它们是否适合您项目中的该目的取决于您的团队和客户的需求。
The tools programmers use to write technology-facing tests can also be used for business-facing tests. Whether they are suited for that purpose in your project depends on the needs of your team and your customers.
在本章中,我们解释了支持团队的面向技术的测试的目的,并讨论了哪些团队需要有效地使用它们。
In this chapter, we explained the purpose of technology-facing tests that support the team, and we talked about what teams need to use them effectively.
支持编程的面向技术的测试让团队产生最高质量的代码;它们构成了所有其他类型测试的基础。
Technology-facing tests that support programming let the team produce the highest quality code possible; they form the foundation for all other types of testing.
这个象限测试的好处包括走得更快和做的更多,但速度和数量永远不应该是最终目标。
The benefits of this quadrant’s tests include going faster and doing more, but speed and quantity should never be the ultimate goal.
程序员编写面向技术的测试来支持团队并通过增强系统的内部质量和可测试性为测试人员提供巨大的价值。
Programmers write technology-facing tests that support the team and provide great value to testers by enhancing the internal quality and testability of the system.
未能实施与敏捷开发相关的核心实践的团队可能会举步维艰。
Teams that fail to implement the core practices related to agile development are likely to struggle.
遗留系统通常是测试驱动开发的最大障碍,但这些问题可以通过增量方法来克服。
Legacy systems usually present the biggest obstacles to test-driven development, but these problems can be overcome with incremental approaches.
如果您的团队现在不进行这些测试,您可以通过让其他团队成员参与并获得管理层的支持来帮助他们开始。
If your team doesn’t now do these tests, you can help them get started by engaging other team members and getting support from management.
面向技术的测试和支持团队的面向业务的测试之间可能存在一些重叠。然而,当面临选择时,将测试推到最低级别以最大化投资回报率。
There can be some overlap between technology-facing tests and business-facing tests that support the team. However, when faced with a choice, push tests to the lowest level in order to maximize ROI.
团队应该建立持续集成、构建和测试流程,以便尽快提供反馈。
Teams should set up continuous integration, build, and test processes in order to provide feedback as quickly as possible.
敏捷团队需要用于源代码控制、测试自动化、IDE 和构建管理等任务的工具,以促进支持团队的面向技术的测试。
Agile teams require tools for tasks such as source code control, test automation, IDEs, and build management in order to facilitate technology-facing tests that support the team.
在上一章中,我们谈到了程序员测试,即那些帮助程序员确保他们编写的代码正确的低级测试。他们怎么知道应该建造什么?在分阶段和门控方法中,我们尝试通过预先收集需求并尽可能详细地说明来解决这个问题。在使用敏捷实践的项目中,我们完全相信客户理解的故事卡和测试,以帮助编写正确的代码。这些“可理解的”测试是本章的主题。
In the last chapter, we talked about programmer tests, those low-level tests that help programmers make sure they have written the code right. How do they know the right thing to build? In phased and gated methodologies, we try to solve that by gathering requirements up front and putting as much detail in them as possible. In projects using agile practices, we put all our faith in story cards and tests that customers understand in order to help code the right thing. These “understandable” tests are the subject of this chapter.
哎呀,我们正在开始一个迭代,其中的信息不多于适合索引卡的信息,如图8-1所示。
Yikes, we’re starting an iteration with no more information than what fits on an index card, something like what’s shown in Figure 8-1.
信息不多,也不是故意的。故事是对所需功能的简要描述,有助于规划和确定工作的优先级。在传统的瀑布项目中,开发团队可能是给定一个冗长的需求文档,其中包括功能集的每个细节。在敏捷项目中,客户团队和开发团队根据故事展开对话。团队需要某种需求,而且他们需要的级别可以让他们几乎立即开始编写工作代码。为此,我们需要将示例转化为测试,以确认客户真正想要什么。
That’s not much information, and it’s not meant to be. Stories are a brief description of desired functionality and an aid to planning and prioritizing work. On a traditional waterfall project, the development team might be given a wordy requirements document that includes every detail of the feature set. On an agile project, the customer team and development team strike up a conversation based on the story. The team needs requirements of some kind, and they need them at a level that will let them start writing working code almost immediately. To do this, we need examples to turn into tests that will confirm what the customer really wants.
这些面向业务的测试解决了业务需求。这些测试有助于提供全局和足够的细节来指导编码。面向业务的测试基于示例表达需求,并使用客户和开发团队都能理解的语言和格式。示例构成了学习每个功能所需行为的基础,我们使用这些示例作为象限 2 中故事测试的基础(见图8-2)。
These business-facing tests address business requirements. These tests help provide the big picture and enough details to guide coding. Business-facing tests express requirements based on examples and use a language and format that both the customer and development teams can understand. Examples form the basis of learning the desired behavior of each feature, and we use those examples as the basis for our story tests in Quadrants 2 (see Figure 8-2).
面向业务的测试也称为“面向客户”、“故事”、“客户”和“验收”测试。“验收测试”一词特别令人困惑,因为它让一些人只想到“用户验收测试”。在敏捷开发的上下文中,验收测试通常指面向业务的测试,但该术语也可以包括来自象限 4 的面向技术的测试,例如客户对系统性能或安全性的标准。在本章中,我们仅讨论通过指导开发和提供快速反馈来支持团队的面向业务的测试。
Business-facing tests are also called “customer-facing,” “story,” “customer,” and “acceptance” tests. The term “acceptance test” is particularly confusing, because it makes some people think only of “user acceptance tests.” In the context of agile development, acceptance tests generally refer to the business-facing tests, but the term could also include the technology-facing tests from Quadrant 4, such as the customer’s criteria for system performance or security. In this chapter, we’re discussing only the business-facing tests that support the team by guiding development and providing quick feedback.
正如我们在前两章中解释的那样,我们呈现这四个象限的顺序与我们可能执行的顺序无关每个象限的活动。象限 2 中面向业务的测试是在编码开始之前为每个故事编写的,因为它们有助于团队了解要编写的代码。与象限 1 中的测试一样,这些测试推动开发,但在更高的层次上。象限 1 的活动确保内部质量,最大限度地提高团队生产力,并最大限度地减少技术债务。象限 2 测试定义和验证外部质量,并帮助我们知道何时完成。
As we explained in the previous two chapters, the order in which we present these four quadrants isn’t related to the order in which we might perform activities from each quadrant. The business-facing tests in Quadrant 2 are written for each story before coding is started, because they help the team understand what code to write. Like the tests in Quadrant 1, these tests drive development, but at a higher level. Quadrant 1 activities ensure internal quality, maximize team productivity, and minimize technical debt. Quadrant 2 tests define and verify external quality, and help us know when we’re done.
第五部分,“生活中的一次迭代”,检查我们在不同象限中执行测试的顺序。
Part V, “An Iteration in the Life,” examines the order in which we perform tests from the different quadrants.
驱动编码的客户测试通常以可执行格式编写,并且是自动化的,因此团队成员可以根据需要经常运行测试,以查看功能是否按预期工作。这些测试或其中的某些子集将成为自动回归套件的一部分,以便未来的开发不会无意中改变系统行为。
The customer tests to drive coding are generally written in an executable format, and automated, so that team members can run the tests as often as they like in order to see if the functionality works as desired. These tests, or some subset of them, will become part of an automated regression suite so that future development doesn’t unintentionally change system behavior.
当我们讨论期望行为的故事和示例时,我们还必须定义非功能性需求,例如性能、安全性和可用性。出色地还要记下手动探索性测试的场景。我们将在有关象限 3 和 4 的章节中讨论这些其他类型的测试活动。
As we discuss the stories and examples of desired behavior, we must also define nonfunctional requirements such as performance, security, and usability. We’ll also make note of scenarios for manual exploratory testing. We’ll talk about these other types of testing activities in the chapters on Quadrants 3 and 4.
我们听到很多关于敏捷团队如何获取需求的问题。我们如何知道我们编写的代码应该做什么?我们如何获得足够的信息来开始编码?如何让客户同声说话,把需求说清楚?我们从哪里开始每个故事?我们如何让客户给我们举个例子?我们如何使用这些来编写故事测试?
We hear lots of questions relating to how agile teams get requirements. How do we know what the code we write should do? How do we obtain enough information to start coding? How do we get the customers to speak with one voice and present their needs clearly? Where do we start on each story? How do we get customers to give us examples? How do we use those to write story tests?
本章解释了我们创建面向业务的测试的策略,这些测试支持团队开发每个故事。让我们从更多地讨论需求开始。
This chapter explains our strategy for creating business-facing tests that support the team as it develops each story. Let’s start by talking more about requirements.
几乎我们所知道的每个开发团队,无论敏捷与否,都在与需求作斗争。传统瀑布项目的团队可能会花费数月的时间来收集需求,结果却发现它们是错误的或很快就过时了。混沌模式下的团队可能根本没有要求,程序员会尽最大可能猜测功能应该如何工作。
Just about every development team we’ve known, agile or not, struggles with requirements. Teams on traditional waterfall projects might invest months in requirements gathering only to have them be wrong or quickly get out of date. Teams in chaos mode might have no requirements at all, with the programmers making their best guess as to how a feature should work.
敏捷开发拥抱变化,但是当需求在迭代过程中发生变化时会发生什么?我们不希望在开始编码之前有很长的需求收集期,但我们如何才能确定我们(和我们的客户)真正了解每个故事的细节?
Agile development embraces change, but what happens when requirements change during an iteration? We don’t want a long requirements-gathering period before we start coding, but how can we be sure we (and our customers) really understand the details of each story?
在敏捷开发中,新特性通常以故事或故事组的形式开始,由客户团队编写。编写故事并不是要弄清楚实现细节,尽管高层讨论会对依赖关系和创建的故事数量产生影响。如果技术团队的某些成员可以参与故事编写会议,这将很有帮助,这样他们就可以输入功能故事并帮助确保将技术故事作为积压工作的一部分。程序员和测试人员还可以帮助客户将故事分解为适当的大小,建议可能更实用的替代方案,并讨论故事之间的依赖关系。
In agile development, new features usually start out life as stories, or groups of stories, written by the customer team. Story writing is not about figuring out implementation details, although high-level discussions can have an impact on dependencies and how many stories are created. It’s helpful if some members of the technical team can participate in story-writing sessions so that they can have input into the functionality stories and help ensure that technical stories are included as part of the backlog. Programmers and testers can also help customers break stories down to appropriate sizes, suggest alternatives that might be more practical to implement, and discuss dependencies between stories.
故事本身并没有提供有关所需功能的太多细节。它们通常只是一个句子,表达了谁想要这个特性,这个特性是什么,以及他们为什么想要它。“作为一名互联网购物者,我需要一种方法来从我的购物车中删除商品,这样我就不必购买不需要的商品”很多想象力。故事仅作为业务专家和开发团队之间持续对话的起点。如果团队成员了解客户试图解决的问题,他们可以提出可能更易于使用和实施的替代方案。
Stories by themselves don’t give much detail about the desired functionality. They’re usually just a sentence that expresses who wants the feature, what the feature is, and why they want it. “As an Internet shopper, I need a way to delete items from my shopping cart so I don’t have to buy unwanted items” leaves a lot to the imagination. Stories are only intended as a starting point for an ongoing dialogue between business experts and the development team. If team members understand what problem the customer is trying to solve, they can suggest alternatives that might be simpler to use and implement.
在客户和开发人员之间的这种对话中,敏捷团队扩展故事,直到他们有足够的信息来编写适当的代码。测试人员帮助引出每个故事的示例和上下文,并帮助客户编写故事测试。这些测试在程序员编写代码时为他们提供指导,并帮助团队了解何时满足了客户的满意度条件。如果您的团队有用例,他们可以帮助补充示例或指导测试以阐明所需的功能(见图8-3)。
In this dialogue between customers and developers, agile teams expand on stories until they have enough information to write appropriate code. Testers help elicit examples and context for each story, and help customers write story tests. These tests guide programmers as they write the code and help the team know when it has met the customers’ conditions of satisfaction. If your team has use cases, they can help to supplement the example or coaching test to clarify the needed functionality (see Figure 8-3).
在敏捷开发中,我们承认我们永远无法提前了解故事的所有需求。在故事测试通过的代码完成后,我们还需要做更多的测试,以更好地了解需求和功能应该如何工作。
In agile development, we accept that we’ll never understand all of the requirements for a story ahead of time. After the code that makes the story tests pass is completed, we still need to do more testing to better understand the requirements and how the features should work.
在客户有机会看到团队正在交付什么之后,他们可能会对他们希望它如何工作有不同的想法。客户通常对他们想要什么有一个模糊的概念,并且很难准确定义那是什么。该团队与客户或客户代理合作进行迭代,可能只提供解决方案的核心。团队在多次迭代中不断完善功能,直到定义并交付该功能。
After customers have a chance to see what the team is delivering, they might have different ideas about how they want it to work. Often customers have a vague idea of what they want and a hard time defining exactly what that is. The team works with the customer or customer proxy for an iteration and might deliver just a kernel of a solution. The team keeps refining the functionality over multiple iterations until it has defined and delivered the feature.
能够迭代是敏捷开发提倡小版本和一次开发一小块的原因之一。如果我们的客户对我们在本次迭代中交付的代码的行为不满意,我们可以在他们认为重要的情况下在下一次快速纠正。需求变更几乎是不可避免的。
Being able to iterate is one reason agile development advocates small releases and developing one small chunk at a time. If our customer is unhappy with the behavior of the code we deliver in this iteration, we can quickly rectify that in the next, if they deem it important. Requirements changes are pretty much inevitable.
我们必须尽可能多地了解客户的需求。如果我们的最终用户在我们的位置工作,或者可以前往他们的位置,我们应该与他们坐在一起,与他们一起工作,并尽可能地完成他们的工作。我们不仅会更好地理解他们的需求,而且我们甚至可能会识别出他们没有想到的需求。
We must learn as much as we can about our customers’ wants and needs. If our end users work in our location, or it’s feasible to travel to theirs, we should sit with them, work alongside them, and be able to do their jobs if we can. Not only will we understand their requirements better but we might even identify requirements they didn’t think to state.
测试需要包括比客户规定的要求更多的内容。我们需要测试后期条件、对整个系统的影响以及与其他系统的集成。我们识别风险并根据需要通过测试减轻风险。所有这些因素都指导着我们的编码。
Tests need to include more than the customers’ stated requirements. We need to test for post conditions, impact on the system as a whole, and integration with other systems. We identify risks and mitigate those with tests as needed. All of these factors guide our coding.
我们还可以使用我们的测试来提供开发团队和业务专家都能理解的通用语言。正如 Brian Marick [2008] 指出的那样,共享语言可以帮助业务人员设想他们想要的功能。它可以帮助程序员精心设计易于扩展的代码。可以表达期望和不期望行为的真实示例,以便业务和技术方面都能理解它们。具有不同背景和观点的人可以访问图片、流程图、电子表格和原型。我们可以使用这些工具来查找示例,然后轻松地将这些示例转化为测试。测试需要以业务用户可以理解的方式编写,但仍可由技术团队执行。
We can also use our tests to provide a common language that’s understood by both the development team and the business experts. As Brian Marick [2008] points out, a shared language helps the business people envision the features they want. It helps the programmers craft well-designed code that’s easy to extend. Real-life examples of desired and undesired behavior can be expressed so that they’re understood by both the business and technical sides. Pictures, flow diagrams, spreadsheets, and prototypes are accessible to people with different backgrounds and viewpoints. We can use these tools to find examples and then easily turn those examples into tests. The tests need to be written in a way that’s comprehensible to a business user reading them yet still executable by the technical team.
面向业务的测试还有助于定义范围,以便每个人都知道什么是故事的一部分,什么不是。许多测试框架现在允许团队创建领域语言并使用该语言定义测试。Fit(集成框架的功能)就是其中之一。
Business-facing tests also help define scope, so that everyone knows what is part of the story and what isn’t. Many of the test frameworks now allow teams to create a domain language and define tests using that language. Fit (Functional for Integrated Framework) is one of those.
有关 Fit 的更多信息,请参见第 9 章“支持团队的面向业务的测试工具包”。
More on Fit in Chapter 9, “Toolkit for Business-Facing Tests that Support the Team.”
可以说我们的客户会向我们提供我们需要的示例,以便我们了解每个故事应该传递的价值。但是,如果他们不知道如何解释他们想要的东西怎么办?在下一节中,我们将建议一些方法来帮助客户定义他们的满意度条件。
It’s fine to say that our customers will provide to us the examples that we need to have in order for us to understand the value that each story should deliver. But what if they don’t know how to explain what they want? In the next section, we’ll suggest ways to help customers define their conditions of satisfaction.
如果您曾经是要求特定软件功能的客户,您就会知道准确表达您想要什么是多么困难。通常,直到您能够看到、感觉到、触摸和使用它时,您才真正确切地知道自己想要什么。我们有很多方法可以帮助我们的客户弄清楚他们想要什么。
If you’ve ever been a customer requesting a particular software feature, you know how hard it is to articulate exactly what you want. Often, you don’t really know exactly what you want until you can see, feel, touch and use it. We have lots of ways to help our customers get clarity about what they want.
从提问开始。测试人员特别擅长提出各种问题,因为他们了解大局,面向业务的和技术方面的故事,并始终考虑最终用户体验。要问的一般问题的类型是:
Start by asking questions. Testers can be especially good at asking a variety of questions because they are conscious of the big picture, the business-facing and technical aspects of the story, and are always thinking of the end user experience. Types of general questions to ask are:
这个故事能解决问题吗?
Is this story solving a problem?
如果是这样,我们要解决的问题是什么?
If so, what’s the problem we’re trying to solve?
我们能否实施无法解决问题的解决方案?
Could we implement a solution that doesn’t solve the problem?
这个故事将如何为企业带来价值?
How will the story bring value to the business?
谁是该功能的最终用户?
Who are the end users of the feature?
他们将从中获得什么价值?
What value will they get out of it?
用户在使用该功能之前和之后会做什么?
What will users do right before and right after they use that feature?
我们怎么知道我们已经完成了这个故事?
How do we know we’re done with this story?
丽莎喜欢问的一个问题是,“可能发生的最糟糕的事情是什么?” 最坏的情况往往会产生想法。它们还帮助我们考虑风险并将我们的测试集中在关键区域。另一个好问题是,“可能发生的最好的事情是什么?” 这个问题通常会生成我们的快乐路径测试,但它也可能会揭示一些隐藏的假设。
One question Lisa likes to ask is, “What’s the worst thing that could happen?” Worst-case scenarios tend to generate ideas. They also help us consider risk and focus our tests on critical areas. Another good question is, “What’s the best thing that could happen?” This question usually generates our happy path test, but it might also uncover some hidden assumptions.
最重要的是,请客户举例说明该功能应该如何工作。假设这个故事是关于从在线购物车中删除商品的。请客户在白板上画出删除功能的外观。他们是否需要任何额外的功能,例如确认步骤,或者有机会保存该项目以备日后取回?如果无法完成删除,他们希望看到什么?
Most importantly, ask the customer to give you examples of how the feature should work. Let’s say the story is about deleting items out of an online shopping cart. Ask the customer to draw a picture on a whiteboard of how that delete function might look. Do they want any extra features, such as a confirmation step, or a chance to save the item in case they want to retrieve it later? What would they expect to see if the deletion couldn’t be done?
示例可以构成我们测试的基础。我们的挑战是捕获可能用业务领域语言表达的示例,作为可以实际执行的测试。有些客户很乐意使用 Fit 或 FitNesse 等测试工具来表达示例,只要他们可以用他们的领域语言编写示例即可。
Examples can form the basis for our tests. Our challenge is to capture examples, which might be expressed in the business domain language, as tests that can actually be executed. Some customers are comfortable expressing examples using a test tool such as Fit or FitNesse as long as they can write them in their domain language.
让我们用一个简单的故事来探究示例和测试之间的区别(见图8-4)。人们经常混淆这两个术语。
Let’s explore the difference between an example and a test with a simple story (see Figure 8-4). People often get confused between these two terms.
一个例子看起来像这样:
An example would look something like this:
一页上有 5 个项目。我想选择 20.25 美元的商品 1 并将其放入购物车。我单击下一页,其中还有 5 个项目。我在该页面上以 5.38 美元选择了第二件商品,并将其放入我的购物车。当我说我完成购物时,它会同时显示来自我购物车中第一页和第二页的商品,总计 25.63 美元
There are 5 items on a page. I want to select item 1 for $20.25 and put it in the shopping cart. I click to the next page, which has 5 more items. I select a second item on that page for $5.38 and put it in my shopping cart. When I say I’m done shopping, it will show both the item from the first page and the item from the second page in my shopping cart, with the total of $25.63
测试可能会有很大不同。我们将使用表 8-1中的 Fit 类型格式来向您展示如何表示测试。
The test could be quite a bit different. We’ll use a Fit type format in Table 8-1 to show you how the test could be represented.
该测试以可执行格式捕获示例。它可能不会使用完全相同的输入,但它封装了示例用户场景。可以编写更多测试用例来测试边界条件、边缘用例和其他场景。
The test captures the example in an executable format. It might not use exactly the same inputs, but it encapsulates the sample user scenario. More test cases can be written to test boundary conditions, edge cases, and other scenarios.
每个示例或测试都有一个观点。不同的人会从他们独特的角度写出不同的测试或例子。我们希望尽可能多地捕捉不同的观点,因此请考虑您的用户。
Each example or test has one point of view. Different people will write different tests or examples from their unique perspectives. We’d like to capture as many different viewpoints as we can, so think about your users.
获得正确的需求是许多不同角色的团队成员可以参与进来提供帮助的领域。业务分析师、主题专家、程序员和客户团队的各个成员都可以做出贡献。考虑其他利益相关者,例如您的生产支持团队。他们有非常独特的视角。
Getting the requirements right is an area where team members in many different roles can jump in to help. Business analysts, subject matter experts, programmers, and various members of the customer team all have something to contribute. Think about other stakeholders, such as your production support team. They have a very unique perspective.
我们经常忘记非功能性需求,例如“系统需要运行多长时间?如果失败会怎样?如果我们有传递消息的中间件,我们是否期望消息足够大以至于我们可能需要考虑传输过程中的丢失?或者它们将是一个恒定的大小?如果几个小时没有交通会怎样?系统需要警告人吗?” 对这些类型的需求进行测试通常属于第 3 和第 4 象限,但我们仍然需要编写测试以确保它们完成。
We often forget about nonfunctional requirements such as “How long does the system need to be up? What happens if it fails? If we have middleware that passes messages, do we expect messages to be large enough that we might need to consider loss during transmission? Or will they be a constant size? What happens if there is no traffic for hours? Does the system need to warn someone?” Testing for these types of requirements usually falls into quadrants 3 and 4, but we still need to write tests to make sure they get done.
客户提供给团队的所有例子都很快加起来。我们真的必须将所有这些都变成可执行的测试吗?只要我们有客户告诉我们代码是否按他们想要的方式工作,就不会。使用纸上原型制作等技术,可以在编写一行代码之前测试设计。
All of the examples that customers give to the team add up quickly. Do we really have to turn all of these into executable tests? Not as long as we have the customers there to tell us if the code is working the way they want. With techniques such as paper prototyping, designs can be tested before a line of code is written.
客户团队和开发团队之间密切、持续的协作是获得示例的关键,客户测试基于这些示例驱动编码。沟通是敏捷的核心价值,我们将在下一节详细讨论它。
Close, constant collaboration between the customer team and the developer team is key to obtaining examples on which to base customer tests that drive coding. Communication is a core agile value, and we talk about it more in the next section.
在理想的世界中,我们的客户每天都可以为我们服务。实际上,许多团队接触到他们的业务专家的机会有限,而且在许多情况下,客户位于不同的地点或时区。尽你所能进行面对面的交谈。当你做不到时,电话会议、电话交谈、电子邮件、即时消息、相机和其他通讯工具将不得不替代。幸运的是,随时可以使用更多工具来促进远程通信。我们听说过一些团队,例如 Weyerhaeuser 的 iLevel 的 Erika Boyer 团队,他们使用可以由远程位置的人员控制的网络摄像头。尽可能靠近以直接对话。
In an ideal world, our customers are available to us all day, every day. In reality, many teams have limited access to their business experts, and in many cases, the customers are in a different location or time zone. Do whatever you can to have face-to face conversations. When you can’t, conference calls, phone conversations, emails, instant messages, cameras, and other communication tools will have to substitute. Fortunately, more tools to facilitate remote communication are available all the time. We’ve heard of teams, such as Erika Boyer’s team at iLevel by Weyerhaeuser, that use webcams that can be controlled by the folks in the remote locations. Get as close to you can to direct conversation.
即使有客户并且通信线路畅通无阻,也需要管理通信。我们想与客户团队的每一位成员交谈,但他们都有不同的观点。如果我们对某项功能的工作方式有多个不同的版本,我们将不知道要编写什么代码。让我们考虑让客户就每个故事的满意度条件达成一致的方法。
Even when customers are available and lines of communication are wide open, communication needs to be managed. We want to talk to each member of the customer team, but they all have different viewpoints. If we get several different versions of how a piece of functionality should work, we won’t know what to code. Let’s consider ways to get customers to agree on the conditions of satisfaction for each story.
如果您的客户团队由来自组织不同部门的人员组成,那么他们之间可能对特定故事的确切意图存在冲突意见。在丽莎的公司,业务发展要产生收入的功能,运营需要减少电话支持电话的功能,财务需要简化会计、现金管理和报告的功能。令人惊讶的是,不同观点的人可以对同一个故事做出如此多的独特解释。
If your customer team consists of people from different parts of the organization, there may be conflicting opinions among them about exactly what’s intended by a particular story. In Lisa’s company, business development wants features that generate revenue, operations wants features that cut down on phone support calls, and finance wants features that streamline accounting, cash management, and reporting. It’s amazing how many unique interpretations of the same story can emerge from people who have differing viewpoints.
产品负责人是 Scrum 中的一个角色。他不仅负责实现预先的清晰度,而且还负责在确定故事优先级时充当“客户代表”。不过有一个缺点。当您通过一个人汇集许多不同观点的需求时,可能会丢失一些东西。理想情况下,开发团队应该与客户团队坐在一起,学习如何完成客户的工作。如果我们充分了解客户的需求以执行其日常任务,我们就有更好的机会开发出能够正确支持这些任务的软件。
A Product Owner is a role in Scrum. He’s responsible not only for achieving advance clarity but also for acting as the “customer representative” in prioritizing stories. There’s a downside, though. When you funnel the needs of many different viewpoints through one person, something can be lost. Ideally, the development team should sit together with the customer team and learn how to do the customer’s work. If we understand the customer’s needs well enough to perform its daily tasks, we have a much better chance of producing software that properly supports those tasks.
无论您的团队选择汇集不同的观点,重要的是只有“客户的一个声音”呈现给团队。
However your team chooses to bring together varying viewpoints, it is important that there is only “one voice of the customer” presented to the team.
我们说产品所有者提供满意的条件。让我们更仔细地看看我们的意思。
We said that product owners provide conditions of satisfaction. Let’s look more closely at what we mean.
整个发行版以及每个功能或故事都有令人满意的条件。验收测试有助于定义故事验收。除非预先同意故事的满足条件,否则您的开发团队无法成功交付业务所需的内容。客户团队需要“用同一个声音说话”。如果您从不同的利益相关者那里得到不同的要求,您可能需要推迟并推迟故事,直到您有一个明确的业务满意度条件列表。请客户代表提供每个故事的最少信息量,以便您可以通过富有成效的对话开始每次迭代。
There are conditions of satisfaction for the whole release as well as for each feature or story. Acceptance tests help define the story acceptance. Your development team can’t successfully deliver what the business wants unless conditions of satisfaction for a story are agreed to up front. The customer team needs to “speak with one voice.” If you’re getting different requirements from different stakeholders, you might need to push back and put off the story until you have a firm list of business satisfaction conditions. Ask the customer representative to provide a minimum amount of information on each story so that you can start every iteration with a productive conversation.
了解客户团队要求的最好方法是与客户面对面交谈。因为每个人都在与“需求”作斗争,所以有一些工具可以帮助客户团队完成每个故事。满意的条件不仅应包括故事所传达的特征,还应包括对更大系统的影响。
The best way to understand the customer team’s requirements is to talk with the customers face to face. Because everyone struggles with “requirements,” there are tools to help the customer team work through each story. Conditions of satisfaction should include not only the features that the story delivers but also the impacts on the larger system.
Lisa 的产品负责人使用清单格式来解决以下问题:
Lisa’s product owner uses a checklist format to sort out issues such as:
业务满意条件
Business satisfaction conditions
对网站、文档、发票、表格或报告等现有功能的影响
Impact on existing functions such as the website, documents, invoices, forms, or reports
法律考虑
Legal considerations
对定期调度流程的影响
The impact on regularly scheduled processes
对 UI 故事模型的引用
References to mock-ups for UI stories
帮助文本,或由谁提供
Help text, or who will provide it
测试用例
Test cases
数据迁移(视情况而定)
Data migration (as appropriate)
需要发生的内部沟通
Internal communication that needs to happen
与业务合作伙伴和供应商的外部沟通
External communication to business partners and vendors
产品所有者使用模板将此信息放在团队的 wiki 上,以便团队成员了解故事和开始编写测试时可以使用它。
The product owner uses a template to put this information on the team’s wiki so that it can be used as team members learn about the stories and start writing tests.
第 9 章,“支持团队的面向业务的测试工具包”包括示例检查表以及其他用于表达需求的工具。
Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” includes example checklists as well as other tools for expressing requirements.
这些条件基于客户团队为故事所做的关键假设和决策。他们通常来自与客户就每个故事的高级验收标准进行的对话。讨论满意度条件有助于识别有风险的假设,并增加团队编写和正确估计完成故事所需的所有任务的信心。
These conditions are based on key assumptions and decisions made by the customer team for a story. They generally come out of conversations with the customer about high-level acceptance criteria for each story. Discussing conditions of satisfaction helps identify risky assumptions and increases the team’s confidence in writing and correctly estimating all of the tasks that are needed to complete the story.
在敏捷开发中,我们一次只关注一个故事。每个故事通常是整个应用程序的一个小组成部分,但它可能会产生很大的连锁反应。一个新故事就像一块小石头掉入应用程序的水中,我们可能不会考虑由此产生的波浪会遇到什么。当我们在每次迭代中专注于少数故事时,很容易忘记大局。
In agile development, we focus on one story at a time. Each story is usually a small component of the overall application, but it might have a big ripple effect. A new story drops like a little stone into the application water, and we might not think about what the resulting waves might run into. It’s easy to lose track of the big picture when we’re focusing on a small number of stories in each iteration.
Lisa 的团队发现列出系统中可能受故事影响的所有部分很有帮助。团队可以检查每个“测试点”以查看它可能生成的需求和测试用例。一个小而无辜的故事可能会产生广泛的影响,它所涉及的应用程序的每个部分都可能呈现出另一个层次的复杂性。您需要了解任何代码更改的所有潜在影响。列一个清单是一个很好的起点。在迭代的前几天,团队可以进一步研究和分析受影响的区域,看看是否需要更多的任务卡来覆盖所有这些区域。
Lisa’s team finds it helpful to make a list of all of the parts of the system that might be affected by a story. The team can check each “test point” to see what requirements and test cases it might generate. A small and innocent story might have a wide-ranging impact, and each part of the application that it touches might present another level of complexity. You need to be aware of all the potential impacts of any code change. Making a list is a good place to start. In the first few days of the iteration, the team can research and analyze affected areas further and see whether any more task cards are needed to cover them all.
看起来很小但会影响系统意外区域的故事可能会反过来咬你一口。如果您的团队忘记考虑所有依赖关系,并且如果新代码与现有功能相交,您的故事可能需要比计划更长的时间才能完成。确保您的故事测试包括实施新功能时不太明显的后果。
Stories that look small but that impact unexpected areas of the system can come back to bite you. If your team forgets to consider all dependencies, and if the new code intersects with existing functionality, your story might take much longer than planned to finish. Make sure your story tests include the less obvious fallout from implementing the new functionality.
第 16 章“开始运行”和第 17 章“迭代启动”举例说明团队何时以及如何计划客户测试并探索每个故事的更广泛影响。
Chapter 16, “Hit the Ground Running,” and Chapter 17, “Iteration Kickoff,” give examples of when and how teams can plan customer tests and explore the wider impact of each story.
花时间确定每个故事提供的核心价值,并找出一种渐进的方法来开发它。计划编写测试、编写代码和测试代码的小增量。这样,您的象限 2 测试可确保您按计划交付最小值。
Take time to identify the central value each story provides and figure out an incremental approach to developing it. Plan small increments of writing tests, writing code, and testing the code some more. This way, your Quadrant 2 tests ensure you’ll deliver the minimum value as planned.
写故事是一件棘手的事情。开发团队在估算新故事时,可能会发现一些故事太大,因此会要求客户团队返回将它们分解成更小的故事。故事也可能太小,可能需要与其他故事结合或简单地视为任务。敏捷开发(包括测试)一次只处理一小部分功能。
Writing stories is a tricky business. When the development team estimates new stories, it might find some stories too big, so it will ask the customer team to go back and break them into smaller stories. Stories can be too small as well, and might need to be combined with others or simply treated as tasks. Agile development, including testing, takes on one small chunk of functionality at a time.
当你的团队开始一个新的项目或主题时,请产品负责人在该主题的第一次迭代之前将所有相关故事带到头脑风暴会议中。让产品负责人和其他感兴趣的利益相关者解释这些故事。您可能会发现某些故事需要细分,或者需要编写额外的故事来填补空白。
When your team embarks on a new project or theme, ask the product owner to bring all of the related stories to a brainstorming session prior to the first iteration for that theme. Have the product owner and other interested stakeholders explain the stories. You might find that some stories need to be subdivided or that additional stories need to be written to fill in gaps.
在您了解每个故事应该传递什么价值以及它如何适应系统的上下文之后,您可以将故事分解成小的、可管理的部分。您可以编写客户测试来定义这些小增量,同时牢记对较大应用程序的影响。
After you understand what value each story should deliver and how it fits in the context of the system, you can break the stories down into small, manageable pieces. You can write customer tests to define those small increments, while keeping in mind the impact on the larger application.
编写指导开发的客户测试的一种聪明的增量方法是从“薄片”开始,它遵循从一端到另一端的快乐路径。识别薄片,也称为“钢丝”或“示踪子弹”,可以在主题级别完成,用于验证整体架构。这条钢丝将所有的部件连接在一起,在它牢固之后,可以添加更多的功能。
A smart incremental approach to writing customer tests that guide development is to start with the “thin slice” that follows a happy path from one end to the other. Identifying a thin slice, also called a “steel thread” or “tracer bullet,” can be done on a theme level, where it’s used to verify the overall architecture. This steel thread connects all of the components together, and after it’s solid, more functionality can be added.
有关探索性测试的更多信息,请参阅第 10 章,“评价产品的面向业务的测试”。
See Chapter 10, “Business-Facing Tests that Critique the Product,” for more about exploratory testing.
我们发现这种策略也适用于故事层面。越早构建端到端路径,就能越早进行有意义的测试、获得反馈、开始自动化测试和探索性测试。从一小部分可以测试的最精简的功能开始。这可以被认为是关键路径。对于用户界面,这可能从简单地从一页导航到下一页开始。我们可以将其展示给客户,看看流程是否有意义。我们可以编写一个简单的自动化 GUI 测试。对于本章开头的免运费门槛故事,我们可以从验证用于汇总订单总额的逻辑开始,并确定它是否符合免费送货条件,无需担心它在 UI 上的外观。我们可以使用 FitNesse 等功能测试工具对其进行自动化测试。
We find this strategy works at the story level, too. The sooner you can build the end-to-end path, the sooner you can do meaningful testing, get feedback, start automating tests, and start exploratory testing. Begin with a thin slice of the most stripped-down functionality that can be tested. This can be thought of as the critical path. For a user interface, this might start with simply navigating from one page to the next. We can show this to the customer and see whether the flow makes sense. We could write a simple automated GUI test. For the free-shipping threshold story at the beginning of this chapter, we might start by verifying the logic used to sum up the order total and determine whether it qualifies for free shipping, without worrying about how it will look on the UI. We could automate tests for it with a functional test tool such as FitNesse.
有关回归测试自动化的更多信息,请参见第 IV 部分,“自动化”。
See Part IV, “Automation,” for more about regression test automation.
在薄片工作后,我们可以为下一个块或功能层编写客户测试,并编写使这些测试通过的代码。现在我们也将收到针对这一小增量的反馈。也许我们添加 UI 来显示结账页面,显示该订单符合免费送货条件,或者添加层来持久更新数据库。我们可以添加到我们为第一遍编写的自动化测试中。这是一个“编写测试——编写代码——运行测试——学习”的过程。如果您这样做,您就会知道您的团队生成的所有代码都能让客户满意并且在每个阶段都能正常工作。
After the thin slice is working, we can write customer tests for the next chunk or layer of functionality, and write the code that makes those tests pass. Now we’ll have feedback for this small increment, too. Maybe we add the UI to display the checkout page showing that the order qualified for free shipping, or add the layer to persist updates to the database. We can add on to the automated tests we wrote for the first pass. It’s a process of “write tests—write code—run tests—learn.” If you do this, you know that all of the code your team produces satisfies the customer and works properly at each stage.
如果为故事编写客户测试的任务看起来令人困惑或难以应付,您的团队可能需要将故事分解成更小的步骤或块。一次一小步地完成故事有助于分散测试工作,这样它就不会被推到迭代的末尾。它还可以让您更好地了解自己的进度,并帮助您知道何时完成——我们将在下一节探讨这个主题。
If the task of writing customer tests for a story seems confusing or overwhelming, your team might need to break the story into smaller steps or chunks. Finishing stories a small step at a time helps spread out the testing effort so that it doesn’t get pushed to the end of the iteration. It also gives you a better picture of your progress and helps you know when you’re done—a subject we’ll explore in the next section.
我们有支持团队的面向业务的测试——这些测试是为确保满足条件而编写的。他们从快乐的道路开始,并表明故事符合预期需要。它们涵盖了各种用户场景,并确保系统的其他部分不会受到不利影响。这些测试已经运行,并且通过了(或者至少他们已经确定了要修复的问题)。
We have our business-facing tests that support the team—those tests that have been written to ensure the conditions of satisfaction have been met. They start with the happy path and show that the story meets the intended need. They cover various user scenarios and ensure that other parts of the system aren’t adversely affected. These tests have been run, and they pass (or at least they’ve identified issues to be fixed).
我们现在完成了吗?我们可能是,但我们还不确定。真正的考验是软件的用户是否可以执行故事应该提供的操作。象限 3 和 4 的活动,例如探索性测试、可用性测试和性能测试,将帮助我们找到答案。现在,我们只需要做一些客户测试,以确保我们已经捕获了所有需求。业务用户或产品负责人是确定每个需求是否已交付的合适人选,因此他们是现阶段进行探索的合适人选。
Are we done now? We could be, but we’re not sure yet. The true test is whether the software’s user can perform the action the story was supposed to provide. Activities from Quadrants 3 and 4, such as exploratory testing, usability testing, and performance testing will help us find out. For now, we just need to do some customer tests to ensure that we have captured all of the requirements. The business users or product owners are the right people to determine whether every requirement has been delivered, so they’re the right people to do the exploring at this stage.
当所有测试都通过并且确定了任何遗漏的需求时,我们就完成了支持程序员寻求执行“正确事情”的代码的目的。这并不意味着我们完成了测试。我们将在接下来的章节中详细讨论这一点。
When the tests all pass and any missed requirements have been identified, we are done for the purpose of supporting the programmers in their quest for code that does the “right thing.” It does not mean we are done testing. We’ll talk much more about that in the chapters that follow.
客户测试的另一个目标是识别高风险区域并确保编写代码以巩固这些区域。风险管理是任何软件开发方法中的基本实践,测试人员在识别和减轻风险方面发挥作用。
Another goal of customer tests is to identify high-risk areas and make sure the code is written to solidify those. Risk management is an essential practice in any software development methodology, and testers play a role in identifying and mitigating risks.
编写客户测试不仅是为了定义代码的预期行为,也是为了管理风险。通过测试推动开发并不意味着我们会预先确定每一个需求或能够在我们完成时完美地预测。它确实让我们有机会识别风险并通过可执行的测试用例来减轻风险。风险分析并不是一项新技术。敏捷开发通过将业务价值优先化为小的、经过测试的可交付部分并让客户参与增量验收,从而从本质上减轻了一些风险。然而,我们仍然应该集思广益了解潜在事件、它们可能发生的可能性以及如果它们确实发生对组织的影响,以便可以采用正确的缓解策略。
Customer tests are written not only to define expected behavior of the code but to manage risk. Driving development with tests doesn’t mean we’ll identify every single requirement up front or be able to predict perfectly when we’re done. It does give us a chance to identify risks and mitigate them with executable test cases. Risk analysis isn’t a new technique. Agile development inherently mitigates some risks by prioritizing business value into small, tested deliverable pieces and by having customer involvement in incremental acceptance. However, we should still brainstorm potential events, the probability they might occur, and the impact on the organization if they do happen so that the right mitigation strategy can be employed.
如果测试是针对不太可能的边缘情况,则对预定义测试进行编码效果不佳。虽然我们不想只测试快乐路径,但它是一个很好的起点。知道快乐路径后,我们可以定义风险最高的场景——不仅有坏结果而且很有可能发生的情况。
Coding to predefined tests doesn’t work well if the tests are for improbable edge cases. While we don’t want to test only the happy path, it’s a good place to start. After the happy path is known, we can define the highest risk scenarios—cases that not only have a bad outcome but also have a good possibility of happening.
除了向客户团队提出诸如“可能发生的最糟糕的事情是什么?”之类的问题之外,还要向程序员提出如下问题:“这部分代码的后置条件是什么?什么应该保存在数据库中?我们应该寻找什么样的行为?” 指定测试以涵盖操作的潜在风险结果。
In addition to asking the customer team questions such as “What’s the worst thing that could happen?,” ask the programmers questions like these: “What are the post conditions of this section of code? What should be persisted in the database? What behavior should we look for down the line?” Specify tests to cover potentially risky outcomes of an action.
程序员可以识别代码的脆弱部分。这个故事是否涉及将遗留代码与新架构拼接在一起?被更改的代码是否与另一个系统交互或依赖于第三方软件?通过与程序员和其他团队成员讨论潜在影响和风险区域,我们可以计划适当的测试活动。
Programmers can identify fragile parts of the code. Does the story involve stitching together legacy code with a new architecture? Does the code being changed interact with another system or depend on third-party software? By discussing potential impacts and risky areas with programmers and other team members, we can plan appropriate testing activities.
还有一个风险。我们可能会提前编写详细的测试用例,以至于团队在树中迷失了森林;也就是说,当我们专注于可能被证明无关紧要的细节时,我们可能会忘记大局。
There’s another risk. We might get so involved writing detailed test cases up front that the team loses the forest in the trees; that is, we can forget the big picture while we concentrate on details that might prove irrelevant.
作为一个敏捷团队,我们在短迭代中工作,因此在我们开始之前对花在编写测试上的时间进行时间限制很重要。每次迭代完成后,花时间评估预先提供更多细节是否有帮助。是否有足够的测试来让团队走上正轨?是否因为故事被误解而浪费了很多时间?Lisa 的团队发现最好在编码之前编写高级故事测试,在编码开始后编写详细的测试用例,然后在代码交付时对其进行探索性测试,以便为团队提供更多信息并帮助进行必要的调整.
As an agile team, we work in short iterations, so it’s important to time-box the time spent writing tests before we start. After each iteration is completed, take the time to evaluate whether more detail up front would have helped. Were there enough tests to keep the team on track? Was there a lot of wasted time because the story was misunderstood? Lisa’s team has found it best to write high-level story tests before coding, to write detailed test cases once coding starts, and then to do exploratory testing on the code as it’s delivered in order to give the team more information and help make needed adjustments.
珍妮特从事的项目有一些非常密集的计算。为了确保计算正确完成,在开始编码之前创建详细示例和测试所花费的时间是值得的。了解领域和每个故事的影响对于评估风险和选择正确的缓解策略至关重要。
Janet worked on a project that had some very intensive calculations. The time spent creating detailed examples and tests before coding started, in order to ensure that the calculations were done correctly, was time well spent. Understanding the domain, and the impact of each story, is critical to assessing the risk and choosing the correct mitigation strategy.
虽然面向业务的测试可以帮助降低风险,但其他类型的测试也很重要。例如,许多最严重的问题通常是在手动探索性测试期间发现的。性能、安全性、稳定性和可用性也是风险的来源。在第 3 和第 4 象限的章节中讨论了减轻这些其他风险的测试。
While business-facing tests can help mitigate risks, other types of tests are also critical. For example, many of the most serious issues are usually uncovered during manual exploratory testing. Performance, security, stability, and usability are also sources of risk. Tests to mitigate these other risks are discussed in the chapters on Quadrants 3 and 4.
试验并找到您的团队可以在使用前期细节和保持专注于大局之间取得平衡的方法。短期敏捷迭代的美妙之处在于,您经常有机会评估您的流程的运作方式,以便您可以进行持续改进。
Experiment and find ways that your team can balance using up-front detail and keeping focused on the big picture. The beauty of short agile iterations is that you have frequent opportunities to evaluate how your process is working so that you can make continual improvements.
当敏捷团队中的程序员准备好进行测试驱动开发时,他们会针对故事使用面向业务的测试,以便知道要编写什么代码。从测试开始意味着每个人都在思考设计代码的最佳方式,以使测试更容易。象限 2 中面向业务的测试表示为自动化测试。它们需要清晰易懂、易于运行并提供快速反馈;否则,它们不会被使用。
When programmers on an agile team get ready to do test-driven development, they use the business-facing tests for the story in order to know what to code. Working from tests means that everyone thinks about the best way to design the code to make testing easier. The business-facing tests in Quadrant 2 are expressed as automated tests. They need to be clearly understood, easy to run, and provide quick feedback; otherwise, they won’t get used.
可以为程序员编写手动测试脚本,让他们在签入代码之前执行,这样他们就可以确保满足客户的条件,但期望他们会长期如此麻烦是不现实的。当必须每两周或每 30 天交付一次有意义的商业价值时,信息必须是直接和自动的。缺乏经验的敏捷团队可能会接受在开发人员测试级别比在客户测试级别更容易通过自动化测试驱动编码的需要等级。然而,如果没有客户测试,程序员就很难知道要编写哪些单元测试。
It’s possible to write manual test scripts for the programmers to execute before they check in code so that they can make sure they satisfied the customer’s conditions, but it’s not realistic to expect they’ll go to that much trouble for long. When meaningful business value has to be delivered every two weeks or every 30 days, information has to be direct and automatic. Inexperienced agile teams might accept the need to drive coding with automated tests at the developer test level more easily than at the customer test level. However, without the customer tests, the programmers have a much harder time knowing what unit tests to write.
每个敏捷团队都必须找到一个流程来编写和自动化面向业务的测试,以推动开发。只对面向技术的测试进行自动化的团队发现,他们可以拥有没有错误的代码,但这些代码并不能满足客户的需求。不自动执行任何测试的团队将背负技术债务。
Each agile team must find a process of writing and automating business-facing tests that drive development. Teams that automate only technology-facing tests find that they can have bug-free code that doesn’t do what the customer wants. Teams that don’t automate any tests will anchor themselves with technical debt.
Part IV, “Test Automation,” will guide you as you develop an automation strategy.
象限 2 包含许多不同类型的测试和活动。我们需要合适的工具来促进收集、讨论和交流示例和测试。如果团队位于同一地点,则简单的工具(如纸张或白板)可以很好地收集示例。更复杂的工具可帮助团队编写面向业务的测试,以可执行、可自动化的格式指导开发。在下一章中,我们将了解引出示例以及编写、交流和执行支持团队的面向业务的测试所需的各种工具。
Quadrant 2 contains a lot of different types of tests and activities. We need the right tools to facilitate gathering, discussing, and communicating examples and tests. Simple tools such as paper or a whiteboard work well for gathering examples if the team is co-located. More sophisticated tools help teams write business-facing tests that guide development in an executable, automatable format. In the next chapter, we’ll look at the kinds of tools needed to elicit examples, and to write, communicate, and execute business-facing tests that support the team.
在本章中,我们研究了在编码过程中通过面向业务的测试来支持团队的方法。
In this chapter, we looked at ways to support the team during the coding process with business-facing tests.
在敏捷开发中,示例和面向业务的测试,而不是传统的需求文档,告诉团队要编写什么代码。
In agile development, examples and business-facing tests, rather than traditional requirements documents, tell the team what code to write.
在短期迭代中处理功能的薄片,使客户有机会查看和使用应用程序并根据需要调整他们的需求。
Working on thin slices of functionality, in short iterations, gives customers the opportunity to see and use the application and adjust their requirements as needed.
测试人员贡献的一个重要领域是帮助客户表达满意条件,并为每个故事创建期望和不期望的行为示例。
An important area where testers contribute is helping customers express satisfaction conditions and create examples of desired, and undesired, behavior for each story.
提出开放式问题以帮助客户思考所有需要的功能并防止隐藏重要假设。
Ask open-ended questions to help the customer think of all of the desired functionality and to prevent hiding important assumptions.
帮助客户就故事的期望行为达成共识,以适应业务不同部门的各种观点。
Help the customers achieve consensus on desired behavior for stories that accommodate the various viewpoints of different parts of the business.
帮助客户开发工具(例如,故事清单)来表达业务满意度等信息。
Help customers develop tools (e.g., a story checklist) to express information such as business satisfaction conditions.
开发和客户团队应该考虑给定故事影响的应用程序的所有部分,同时牢记整个系统的功能。
The development and customer teams should think through all of the parts of the application that a given story affects, keeping the overall system functionality in mind.
与您的团队合作,将功能集分解为小的、可管理的故事和故事中的路径。
Work with your team to break feature sets into small, manageable stories and paths within stories.
循序渐进地遵循“编写测试—编写代码—运行测试—学习”的模式,在每次通过功能的基础上进行构建。
Follow a pattern of “write test—write code—run tests—learn” in a step-by-step manner, building on each pass through the functionality.
使用测试和示例来降低缺少功能或忽视全局的风险。
Use tests and examples to mitigate risks of missing functionality or losing sight of the big picture.
通过面向业务的测试驱动编码使开发团队不断意识到实现可测试应用程序的必要性。
Driving coding with business-facing tests makes the development team constantly aware of the need to implement a testable application.
支持团队的面向业务的测试必须自动化,以便快速轻松地提供反馈,以便团队可以在短迭代中交付价值。
Business-facing tests that support the team must be automated for quick and easy feedback so that teams can deliver value in short iterations.
在上一章中,我们讨论了如何进行业务或功能测试以支持团队努力构建正确的软件。在本章中,我们将研究一些可以用来帮助您的团队在象限 2 测试中取得成功的工具。
In the previous chapter, we talked about how to approach business or functional testing to support the team in its effort to build the right software. In this chapter, we’ll examine some of the tools you can use to help your team succeed with Quadrant 2 tests.
我们如何捕获面向业务的测试来帮助程序员知道要编写什么代码?程序员和客户之间面对面的对话通常是最好的方式,但即使客户是您团队的一员,他们也没有整天与程序员闲逛并解释功能。如果任何客户或开发人员团队成员在不同的位置,即兴的走廊对话可能不可行。此外,六个月从现在开始,我们可能想要一种方法来记住我们为什么以某种方式对功能进行编码。如果我们的一些团队成员在不同的地点,我们肯定需要某种方式以电子方式共享信息。
How do we capture the business-facing tests that help the programmers know what to code? Face-to-face conversations between programmers and customers are usually the best way, but even when customers are part of your team, they don’t have all day to hang out with programmers and explain features. If any customer or developer team members are in different locations, impromptu hallway conversations might not be feasible. Besides, six months from now, we might want a way to remember why we coded a piece of functionality a certain way. If some of our team members are in different locations, we’re definitely going to need some way to share information electronically.
随着敏捷开发的流行,我们有越来越多的工具来帮助我们捕获示例并使用它们编写可执行的测试。可用的工具变化太快,我们无法在本书中列出它们的清单,但我们可以提供一些工具示例和一些使用它们的策略,以帮助提供支持团队开发新故事的面向业务的测试。我们在这里讨论的一些工具不是新的,也不是专门针对敏捷开发的,但它们在敏捷项目中运行良好。
As agile development has gained in popularity, we have more and more tools to help us capture examples and use them to write executable tests. The tools available are changing too fast for us to include an inventory of them in this book, but we can offer some examples of tools and some strategies for using them to help provide business-facing tests that support the team’s development of new stories. Some of the tools we discuss here aren’t new, or specific to agile development, but they work well in an agile project.
选择所需工具的策略应基于团队的技能组合、应用程序使用的技术、团队的自动化优先级、时间和预算限制以及您的情况所特有的其他问题。您对一个或多个工具的选择不应基于推销员提供的最新最酷的工具。您可能需要许多不同的工具来解决不同的问题。
Your strategy for selecting the tools you need should be based on your team’s skill set, the technology your application uses, your team’s automation priorities, time and budget constraints, and other concerns unique to your situation. Your selection of a tool or tools should not be based on the latest and coolest tool offered by a salesman. You might need many different tools to solve different problems.
有关测试自动化的一般方法的更多信息,请参阅第 14 章,“敏捷测试自动化策略”。
For more information about a general approach to test automation, see Chapter 14, “An Agile Test Automation Strategy.”
我们鼓励客户做一些提前准备,并准备好在迭代计划期间为每个故事解释示例。测试人员可以很好地帮助客户弄清楚如何在迭代开始时提供正确数量的细节。很难取得恰到好处的平衡。
We encourage customers to do some advance preparation and to be ready to explain examples for each story during iteration planning. Testers are in a good position to help customers figure out how to provide the right amount of detail at the beginning of the iteration. It’s hard to strike just the right balance.
在测试用例中试验不同级别的前期细节,找出最适合您的团队的方法。无论您追求何种程度的详细信息,您都需要一些方法来帮助客户找到并表达所需系统行为的示例。在下一节中,我们将了解可以执行此操作的工具类型。
Experiment with different levels of up-front detail in test cases to figure out what works best for your team. Whatever level of detail you’re after, you need some way to help customers find and express examples of desired system behavior. In the next section, we look at the types of tools that can do that.
正如我们在第 8 章中指出的那样,故事只是就所需行为进行长时间对话的起点。拥有正确大小的故事,其中明确说明了功能、用户和目的,这让我们有一个良好的开端。它们不是很详细,因为正如 Mike Cohn [ 2004 ] 指出的那样,最好推迟收集细节,直到故事包含在迭代中。为一个可能永远不会被收录的故事收集细节是一种资源浪费。我们喜欢 Mike Cohn 在User Stories Applied中描述的用户故事的“角色、功能、业务价值”模式,如下所示:
As we pointed out in Chapter 8, stories are only a starting place for a prolonged conversation about the desired behavior. Having correctly sized stories where the feature, user, and purpose are clearly stated gives us a head start. They aren’t very detailed, because as Mike Cohn [2004] points out, it’s best to defer collecting details until the story is included in an iteration. Collecting details for a story that might never be included is a waste of resources. We like the “role, function, business value” pattern for user stories that Mike Cohn describes in User Stories Applied, as in:
作为一个(角色),我想要(功能)以便(商业价值)。
As a (role), I want (function) so that (business value).
这种格式并不适合所有人,因此我们鼓励您进行试验,看看哪种格式最适合您的情况。无论您的用户故事如何阅读,您都需要某种方式通过示例和面向业务的测试来指导开发来充实这些故事。
This format doesn’t work for everyone, so we encourage you to experiment and see what works best in your situation. Regardless of how your user stories read, you need some way to flesh those stories out with examples and business-facing tests that guide development.
一个简单的故事可以产生广泛的影响,不仅对应用程序,而且对整个组织、它的客户、它的同事、供应商或合作伙伴。如果我们更改 API,我们必须通知可能正在使用它的任何客户或供应商。如果我们计划更改 UI,我们希望,甚至可能有合同义务,提前向用户发出一定数量的通知。故事可能会影响法律问题或影响外部报告。新功能通常意味着新的或更新的文档。当然,更改的功能可能会影响系统的其他部分。
One simple story can have a wide-ranging impact, not only on the application, but across the organization, its clients, its associates, vendors, or partners. If we change an API, we have to notify any customers or vendors who might be using it. If we plan a UI change, we want, or might even be contractually obligated, to give a certain amount of advance notice to users. Stories may affect legal concerns or impact external reporting. New features often mean new or updated documentation. Of course, changed functionality is likely to affect other parts of the system.
包括测试人员在内的软件开发团队应该帮助客户捕获和传达与每个故事或主题相关的所有需求。开发新功能,只是因为法律原因或因为业务合作伙伴没有及时通知而被阻止发布它们,这是一种令人沮丧的时间浪费(问问 Lisa!)。精益开发教会我们在开发软件时避免浪费。
The software development team, including the testers, should help the customer capture and communicate all of the requirements related to each story or theme. Developing new features, only to be prevented from releasing them for legal reasons or because a business partner wasn’t informed in time, is a frustrating waste of time (just ask Lisa!). Lean development teaches us to avoid waste while we develop software.
哪些工具可以帮助我们通过示例说明所需的行为、集思广益潜在的实施和连锁反应,以及创建可以转化为测试的需求?一些例子是:
What tools can help us illustrate desired behavior with examples, brainstorm potential implementations and ripple effects, and create requirements we can turn into tests? Some examples are:
该列表包括许多简单的工具,它们不是敏捷测试所独有的,但不应被忽视。在敏捷开发中,简单的解决方案通常是最好的。让我们更详细地看看这些。
The list includes a number of simple tools that aren’t unique to agile testing but that shouldn’t be neglected. In agile development, simple solutions are usually best. Let’s look at these in more detail.
清单是产品所有者确保他们正确评估和传达故事所有方面的一种方式。Lisa 团队的产品负责人 Steve Perkins 提出了他自己的“故事清单”,以确保他和利益相关者仔细考虑受故事影响的一切。为此,他在团队 wiki 上创建了一个模板。清单指定了满意的条件——业务需要从故事中得到什么。它还包括对现有功能的影响,例如网站、文件、管理表格、账户报表和系统的其他组件以及业务的日常运营。清单确保团队不会因为忘记考虑而错过诸如数据迁移、通知、法律考虑以及与供应商和业务合作伙伴的沟通等要求。图 9-1显示了一个示例故事清单。
Checklists are one way for product owners to make sure they correctly assess and communicate all of the aspects of a story. The product owner for Lisa’s team, Steve Perkins, came up with his own “story checklist” to make sure he and the stakeholders think through everything affected by the story. He created a template on the team wiki for this purpose. The checklist specifies the conditions of satisfaction—what the business needs from the story. It also includes impacts on existing functions such as the website, documents, administrative forms, account statements, and other components of the system and the daily operation of the business. The checklist makes sure the team doesn’t miss requirements such as data migration, notifications, legal considerations, and communications to vendors and business partners because they forgot to consider them. Figure 9-1 shows a sample story checklist.
思维导图是一种简单但有效的方法,可以用来找出您在简单的头脑风暴会议中可能不会想到的想法。思维导图是为表示与中心关键概念相关联的概念、词语或想法而创建的图表。我们使用思维导图来组织这本书。
Mind maps are a simple but effective way to search out ideas that that might not occur to you in a simple brainstorming session. Mind maps are diagrams created to represent concepts, words, or ideas linked to a central key concept. We used mind maps to organize this book.
无论您是购买我们使用的工具还是在白板或大纸上绘图,这都无关紧要。效果是一样的。思维导图使您能够以与您思考问题的方式一致的方式产生想法和工作。
It really doesn’t matter whether you purchase a tool such as the one we used or draw on a whiteboard or a big piece of paper. The effect is the same. Mind maps enable you to generate ideas and work in a way that is consistent with the way you think about problems.
举个例子怎么样?我们正在讨论图 9-2中所示的故事。
How about an example? We’re discussing the story shown in Figure 9-2.
我们聚集在白板周围,开始提问。删除的项目应该去哪里?它们应该被保存以供以后批准,还是应该消失?删除项目后屏幕应该是什么样的?图 9-3显示了我们可能在白板上绘制的思维导图类型的示例。
We gather around the whiteboard and start asking questions. Where should the deleted items go? Should they be saved for later approval, or should they just disappear? What should the screen look like after we delete an item? Figure 9-3 shows an example of the sort of mind map we might draw on a whiteboard.
如果可能,用于指定面向业务的测试的工具应该适合您的业务领域。例如,电子表格被金融服务公司广泛使用,因此对于金融服务领域的项目,使用电子表格来定义故事应该提供的功能示例是有意义的。
When possible, tools for specifying business-facing tests should fit well with your business domain. For example, spreadsheets are widely used by financial services companies, so for a project in the financial services area it makes sense to use spreadsheets to define examples of the functionality that a story should deliver.
客户可以编写一些高级测试用例来帮助在迭代开始之前完善故事,可能使用某种类型的检查表。一些客户团队只是在每张故事卡的背面写几个测试,可能是一个快乐的路径和一个负面的测试。有些人会在电子表格或他们喜欢使用的任何格式中编写更详细的示例。
Customers can write a few high-level test cases to help round out a story prior to the start of the iteration, possibly using some type of checklist. Some customer teams simply write a couple of tests, maybe a happy path and a negative test, on the back of each story card. Some write more detailed examples in spreadsheets or whatever format they’re comfortable working with.
Lisa 团队的产品负责人 Steve Perkins 经常在电子表格中说明复杂的计算和算法,团队可以稍后将其转化为测试。图 9-4显示了他的一个工作表,该工作表对输入值执行计算以生成 ADR 和 ACR 列中的值。这种格式很容易进入自动化测试框架(相应的 FitNesse 示例参见图 9-8 )。
Steve Perkins, the product owner for Lisa’s team, often illustrates complex calculations and algorithms in spreadsheets, which the team can turn into tests later. Figure 9-4 shows one of his worksheets, which performs calculations on the input values to produce the values in the ADR and ACR columns. This format is easy to get into an automated test framework (refer to Figure 9-8 for the corresponding FitNesse example).
查看您的业务专家已经使用的工具,看看它们是否可以调整以记录所需功能行为的示例,以帮助开发团队更好地理解故事。
Look at tools already used by your business experts and see whether they can be adapted to document examples of desired feature behavior to help the development team better understand the story.
Janet 曾与多个使用电子表格作为 Fit 测试输入的团队合作。这允许客户使用他们熟悉的工具工作,但不会浪费任何精力将它们转换为自动化工具。
Janet has worked with several teams that have used spreadsheets as input into their Fit tests. This allows customers to work in a tool that is familiar to them but not waste any effort in translating them to an automation tool.
模型可以采用多种形式。纸质原型是测试屏幕如何协同工作的一种简单但有效的方法。在白板上画图可以达到同样的目的,但不能传来传去。现有应用程序的屏幕截图可以构成讨论如何添加新功能以及新功能适合 UI 的位置的基础。您可能在其他开发方法中使用过此类工具。敏捷开发的最大区别在于我们在开始编写代码时创建和讨论模型,而不是提前数周或数月。我们可以确信模型代表了客户现在的需求。
Mock-ups can take many forms. Paper prototypes are a simple but effective way to test how screens will work together. Drawing on a whiteboard can accomplish the same goal, but it can’t be passed around. Screenshots from existing applications can form the basis of a discussion about how to add a new feature and where it will fit into the UI. You may have used tools like these in other development methodologies. The big difference in agile development is that we create and discuss the mock-ups just as we’re about to start writing the code, rather than weeks or months beforehand. We can be confident that the mock-up represents what the customers want right now.
请参阅第 8 章,了解 Gerard Meszaros 对使用纸质原型和绿野仙踪测试的描述。
See Chapter 8 for Gerard Meszaros’ description of using paper prototypes and Wizard of Oz testing.
图 9-5显示了 Lisa 的团队用来模拟新报告的模型示例——只需标记一个相似的现有报告即可。
Figure 9-5 shows an example of a mock-up that Lisa’s team used to mock up a new report—simply by marking up an existing report that’s similar.
模型不需要花哨或漂亮,也不需要花费很多时间来创建。它们确实需要让客户和开发团队都能理解。
Mock-ups don’t need to be fancy or pretty, or to take a lot of time to create. They do need to be understandable to both the customer and developer teams.
简单的图表工具很有用,无论团队是否在同一地点。以更永久的形式捕获讨论期间制定的工作流或决策树通常是个好主意。流程图可以成为用户场景的基础,可以帮助您将两个或三个用户故事联系在一起。让我们再看看我们在第 8 章中介绍的装运订单故事(见图9-6)。
Simple diagramming tools are helpful, whether the team is co-located or not. It’s often a good idea to capture in a more permanent form a workflow or decision tree worked out during a discussion. Flow diagrams can become the basis of a user scenario that might help you tie two or three user stories together. Let’s look at the shipping order story again that we introduced in Chapter 8 (see Figure 9-6).
图 9-7显示了一个非常简单的决策过程流程图,该过程根据阈值订单金额确定客户的订单是否符合免费送货条件。因为我们已经与我们的客户讨论过这个故事,所以我们发现客户的订单不仅必须超过美元金额阈值,而且必须只发送到一个地址,而且它的重量必须小于运输重量阈值。如果满足所有这些条件,客户的订单将免费发货;否则,客户将不得不从“选择运输选项”页面进行选择。
Figure 9-7 shows a very simple flowchart of a decision process for whether a customer’s order is eligible for free shipping based on a threshold order amount. Because we’ve discussed this story with our customer, we’ve found out that the customer’s order must not only exceed a threshold dollar amount but also must be to one address only, and it must weigh less than a shipping weight threshold. If all of these conditions are satisfied, the customer’s order will ship free; otherwise, the customer will have to select from the “choose shipping options” page.
流程图和思维导图等视觉效果是描述故事功能概览的好方法,尤其是当它们由一组客户、程序员和测试人员绘制时。在敏捷开发中,我们在开始编写测试和代码时创建这些图表。根据这些,团队可以立即开始深入挖掘详细需求。
Visuals such as flow diagrams and mind maps are good ways to describe an overview of a story’s functionality, especially if they’re drawn by a group of customers, programmers, and testers. In agile development, we create these diagrams as we’re about to start writing tests and code. From these, the team can immediately start digging down to the detailed requirements.
如果我们与客户处于不同的位置,我们需要工具来帮助我们与他们交谈。分布式团队告诉我们,桌面共享是帮助他们处理在不同地点工作的首要工具。Windows NetMeeting 和 VNC 是让两个团队成员在不同位置进行配对测试的工具示例。WebEx 和 Skype 等视频会议工具支持远程团队和客户之间的协作和演示。Scriblink 等在线白板和 Mimeo 等交互式白板工具有助于分布式白板讨论。
If we’re in a different location than our customers, we need tools to help us converse with them. Distributed teams tell us that desktop sharing is the number one tool that helps them deal with working in separate locations. Windows NetMeeting and VNC are examples of tools that let two team members in different locations pair-test. Video conferencing tools such as WebEx and Skype enable collaboration and demos between remote teams and customers. Online whiteboards such as Scriblink and interactive whiteboard tools such as Mimeo facilitate distributed whiteboard discussions.
更多适合产品所有者和业务专家直接使用的工具正在变得可用,并且许多团队开发了自己的工具。Fit(集成测试框架)和 FitNesse 等工具旨在促进客户与开发团队之间的协作和沟通。我们听说有更多团队的客户实际使用诸如此类的工具编写测试。
More tools that are geared for direct use by product owners and business experts are becoming available, and many teams develop their own. Tools such as Fit (Framework for Integrated Tests) and FitNesse were designed to facilitate collaboration and communication between the customer and development teams. We’re hearing about more teams where the customers actually write the tests in a tool such as those.
一些团队构建自己的框架,允许客户、业务分析师和测试人员记录可以直接转化为可执行测试的示例。这些通常基于开源工具,例如 xUnit、Fit、Selenium 和 Watir。我们喜欢这种方法,因为它可以节省时间和资源。当您在短迭代中交付生产就绪代码时,您需要一个简化的流程。
Some teams build their own frameworks that allow customers, business analysts, and testers to document examples that can be directly turned into executable tests. These are often based on open source tools such as xUnit, Fit, Selenium, and Watir. We like this approach, because it saves time and resources. When you’re delivering production-ready code in short iterations, you need a streamlined process.
在线论坛工具是电子邮件对话的一个很好的替代方案,可以持续讨论功能或技术问题,尤其是对于并非所有人都坐在一起的团队。电子邮件经常被遗漏或丢失,人们必须记住选择“全部回复”,而且以后很难整理讨论的细节。Lisa 的团队使用在线论坛来征求对不同工具的意见,为功能提出不同的行为,并进行哲学讨论,例如是否跟踪缺陷。
Online forum tools are a good alternative to email conversations for ongoing discussions about features or technical concerns, especially for teams that don’t all sit together. Emails often get missed or lost, people have to remember to choose “Reply all,” and it can be hard to put together the details of the discussion later. Lisa’s team uses an online forum to elicit opinions about different tools, propose different behavior for features, and conduct philosophical discussions such as whether to track defects.
找到合适的电子工具对于分布式团队尤为重要。即时消息、电话、VoIP 和 Skype 帮助我们进行交流,但它们缺乏可视化组件。一些全球团队要求他们的成员在非标准时间开会,以便他们可以进行实时对话,但书面和视觉交流的框架仍然至关重要。
Finding the right electronic tools is particularly vital for distributed teams. Instant messaging, the telephone, VoIP, and Skype help us communicate, but they lack the visual component. Some global teams ask their members to meet at nonstandard hours so that they can have real-time conversations, but frameworks for written and visual communication are still critical.
Wiki 是一种常用工具,用于加强交流和记录讨论和决策。Wiki 使用户能够在 Web 浏览器中编辑网页内容。用户可以添加超链接并轻松创建新页面。您可以上传白板绘图的模型、示例和图片,并使它们在 Wiki 页面上易于显示。分层组织可能难以维护,但是有许多可用的开源和供应商 wiki 软件包可以使管理您的知识库和共享信息更容易管理。如果您的 wiki 知识库已经发展到难以找到任何东西的地步,请聘请技术作家将其转化为有条理的可用文档。
Wikis are a common tool used to enhance communication and record discussions and decisions. Wikis enable users to edit web page content in a web browser. Users can add hyperlinks and easily create new pages. You can upload mock-ups, samples, and pictures of whiteboard drawings and make them easily visible on Wiki pages. The hierarchical organization can get tricky to maintain, but there are lots of open source and vendor wiki software packages available that make managing your knowledgebase and sharing information easier to administer. If your wiki knowledgebase has grown to the point where it’s hard to find anything, hire a technical writer to transform it into organized, usable documentation.
开源和商业工具提供了让团队在线协作处理需求和测试用例的方法。我们怎么强调都不够强调您需要确定可能有用的工具,对它们进行几次迭代试验,并确定它们对您的工作效果如何。您团队的需求会随着时间而改变,因此请始终乐于尝试新技术和框架。
Open source and commercial tools provide ways to let teams collaborate on requirements and test cases online. We can’t emphasize enough the need for you to identify tools that might be helpful, to experiment with them for a few iterations, and to decide how well they work for you. Your team’s needs will change with time, so always be open to trying new techniques and frameworks.
这些工具有助于创建关于故事的对话。借助这些技术,以及尽可能多的实时对话和视觉共享,我们可以从一开始就定义正确的产品。
These tools help create the conversation about the story. With these techniques, and as much real-time conversation and visual sharing as we can manage, we can define the right product from the get-go.
测试工具呢?我们喜欢 Fit 和 FitNesse 等工具固有的协作功能。然而,在我们看来,任何能吸引测试人员和程序员的工具,程序员和客户,以及测试人员和客户交谈是一件很棒的事情。我们知道客户实际使用 Fit、FitNesse、Expect 或其他工具编写测试的团队。当工具的设置方式对每个编写测试的人来说都很清楚,并且域语言易于理解并提供适当的固定装置时,这就会起作用。
What about test tools? We like the collaboration inherent with tools such as Fit and FitNesse. However, in our opinion, any tool that gets testers and programmers, programmers and customers, and testers and customers talking is a great one. We know teams where customers actually write tests in Fit, FitNesse, Expect, or other tools. This works when the tool has been set up in a manner that’s clear to everyone writing tests, with the domain language easy to understand and the appropriate fixtures provided.
有许多开源工具可让您在 GUI 下或 API 层进行测试。我们仅列出了几个,但您的团队需要为您确定合适的工具。
There are a multitude of open source tools that enable you to test below the GUI or at the API layer. We are listing just a few, but your team will need to determine the right tool for you.
一些团队使用 xUnit 工具(如 JUnit 或 NUnit)进行面向业务的测试和面向技术的测试。如果测试人员和客户对这些工具感到满意,并且它们提供了所需的 GUI 背后的所有功能测试,那么它们就很好。为了使这些工具对客户更加友好,团队可能会在单元级工具之上构建一个框架,测试人员和客户可以使用该框架来指定测试。
Some teams use the xUnit tools such as JUnit or NUnit for business-facing tests as well as technology-facing tests. If the testers and customers are comfortable with these tools, and they provide for all of the functional testing behind the GUI needed, they’re fine. To make these tools more customer-friendly, teams might build a framework on top of the unit-level tools that testers and customers can use to specify tests.
Janet 开发过几个这样的应用程序。一个是在组织中部署的消息处理系统。程序员使用 JUnit 进行所有组件和集成测试。他们构建了一个可以使用 JUnit 测试的负载测试框架,因此不需要其他测试工具。GUI 前端非常小,Janet 能够手动测试它。在这种情况下,自动化 GUI 测试毫无意义。
Janet has worked on a couple of applications like that. One was a message handling system that was being deployed in an organization. The programmers used JUnit for all of the component and integration testing. They built a load test framework that could make use of the JUnit tests, so no other testing tools were needed. The GUI front end was so small that Janet was able to test it manually. It made no sense to automate the GUI testing in this case.
行为驱动开发 (BDD) 工具也适用于此目的,因为它们使用更自然的语言来指定测试。行为驱动开发是测试驱动开发的变体,由 Dan North [ 2006 ] 首创,并由许多其他人发展而来。它与领域驱动设计有关,关注领域而不是技术,并用模型驱动设计。BDD 没有使用“测试”或“断言”一词,而是使用了“应该”一词。通过从行为的角度考虑,在代码之前编写规范是很自然的。测试规范使用特定领域的语言来提供客户可以阅读但也可以轻松实现自动化的测试。
Behavior-driven development (BDD) tools are also suited to this purpose, because they use a more natural language for specifying the tests. Behavior-driven development is a variation of test-driven development, pioneered by Dan North [2006], and evolved by many others. It’s related to domain-driven design, with a focus on the domain rather than on the technology, and driving design with a model. Instead of the word “test” or “assert,” BDD uses the word “should.” By thinking in terms of behavior, it’s natural to write specifications ahead of code. Test specifications use a domain-specific language to provide tests that customers can read but that can also be easily automated.
在撰写本文时可用的许多 BDD 工具中的一些包括用于 Java 平台的 easyb 和 JBehave,用于 .NET 的 NBehave 和 NSpec,以及用于 Ruby 的 RSpec。这些工具,如 XUnit 工具,旨在供程序员用来指导编码,但它们也可用于表达推动开发的面向业务的测试,让客户更密切地参与开发过程。
Some of the many BDD tools available as of this writing include easyb and JBehave for the Java platform, NBehave and NSpec for .NET, and RSpec for Ruby. These tools, like the XUnit tools, are intended for use by programmers to guide coding, but they can also be used to express business-facing tests that drive development, involving customers more closely in the development process.
支持团队的面向业务的测试的目标是促进客户和开发人员之间的沟通和协作,并使团队能够在每次迭代中交付真正的价值。一些团队使用单元级工具在这方面做得最好,而其他团队则更好地适应功能级测试工具。
The goal of business-facing tests that support the team is to promote communication and collaboration between customers and developers, and to enable teams to deliver real value in each iteration. Some teams do this best with unit-level tools, and others adapt better to functional-level test tools.
在 Lisa 加入她的第一个敏捷团队之前,“在 GUI 后面”进行测试是一个听起来不错的概念,但她从未有机会尝试过。Fit 和建立在 Fit 之上的 FitNesse 是功能测试工具,源于客户团队需要能够编写和理解推动开发的面向业务的测试。使用这些工具,团队可以在不涉及表示层的情况下测试业务逻辑。
Before Lisa joined her first agile team, testing “behind the GUI” was a concept that sounded good, but she’d never had the opportunity to try it. Fit, and FitNesse, which is built on top of Fit, are functional test tools that grew from the need for the customer team to be able to write and understand the business-facing tests that drive development. With these tools, teams can test business logic without involving the presentation layer.
Fit 和 FitNesse。Fit (Framework for Integrated Tests) 是一个促进协作的开源测试框架,这使其成为帮助细化需求的好工具。Fit 是 Ward Cunningham 的发明,享有杰出的开发人员名单。Fit 使客户、测试人员和程序员能够使用示例来指定他们期望系统执行的操作。测试运行时,Fit 会自动将客户的期望值与实际结果进行比较。
Fit and FitNesse. Fit (Framework for Integrated Tests) is an open source testing framework that promotes collaboration, which makes it a good tool to help refine requirements. The invention of Ward Cunningham, Fit has enjoyed an illustrious roster of contributing developers. Fit enables customers, testers, and programmers to use examples to specify what they expect the system to do. When the tests run, Fit automatically compares customers’ expectations to actual results.
借助 Fit,客户可以使用他们的主题专业知识提供指导,以定义程序员可以编码的示例。程序员通过编写对示例进行实际检查的固定装置来参与。这些装置使用示例中指定的数据与实际程序一起运行。
With Fit, customers can provide guidance using their subject matter expertise to define the examples that the programmers can code against. The programmers participate by writing the fixtures that do the actual checks against the examples. These fixtures use the data specified in the examples to run with the actual program.
适合性测试由将测试输入传递给生产代码然后接受输出并与预期结果进行比较的夹具自动执行。测试结果用颜色编码,因此很容易发现故障或异常。
Fit tests are automated by fixtures that pass the test inputs to the production code and then accept the outputs, which it then compares with expected results. The test results are color-coded, so it’s easy to spot a failure or exception.
Fit 测试以 HTML 表格的形式编写,但团队可以自定义 Fit,以便可以将测试编写为电子表格或客户、测试人员和分析师认为可用的任何形式。
Fit tests are written as HTML tables, but teams can customize Fit so that tests can be written in spreadsheets or whatever form the customers, testers, and analysts find usable.
在fit.c2.com上了解有关 Fit 的更多信息。
Learn more about Fit at fit.c2.com.
FitNesse 是一个基于 Fit 的 Web 服务器、wiki 和软件测试工具。最初由 Robert C. “Uncle Bob” Martin 和 Micah Martin 开发,它是一个拥有活跃开发者社区的开源工具。FitNesse 和 Fit 之间的主要区别是 FitNesse 测试是用 wiki 标记而不是 HTML 表格编写的,一些用户认为后者更容易。它还支持在电子表格中创建测试并将其导入到测试中。
FitNesse is a web server, a wiki, and a software testing tool that is based on Fit. Originally developed by Robert C. “Uncle Bob” Martin and Micah Martin, it’s an open source tool with an active developer community. The main difference between FitNesse and Fit is that FitNesse tests are written in wiki markup instead of HTML tables, which some users find easier. It also supports creating tests in spreadsheets and importing those into the tests.
在www.fitnesse.org上了解有关 FitNesse 的更多信息。
Learn more about FitNesse at www.fitnesse.org.
图 9-8显示了从图 9-4中的示例构建的 FitNesse 测试的一部分。添加了更多输入以使生产代码运行,但基本测试数据来自电子表格。测试结果通过时颜色编码为绿色,失败时为红色。
Figure 9-8 shows part of the FitNesse test that was built from the example in Figure 9-4. More inputs were added to make the production code run, but the essential test data is from the spreadsheet. The test results are color-coded green when they pass, red when they fail.
Fit 或 FitNesse 类型的工具的另一个好处是它促进不同团队成员之间的协作,以便提出正确的测试来指导发展。客户、程序员、测试人员和其他人一起工作来指定和自动化测试。
Another benefit of a Fit or FitNesse type of tool is that it promotes collaboration among different team members in order to come up with the right tests to guide development. Customers, programmers, testers, and others work together to specify and automate the tests.
测试网络服务。Web 服务只是 API 的另一种形式,它使其他应用程序能够访问您的应用程序。让我们谈谈可用于测试系统各种输入的一些工具。
Testing Web Services. Web services is just another form of an API that enables other applications to access your application. Let’s talk about some of the tools you can use to test various inputs into your system.
交叉检查。CrossCheck 是一种用于测试 Web 服务的工具示例。您提供WSDL(Web 服务描述语言);CrossCheck 编译该页面,然后为您呈现一个选项卡式菜单,其中包含供您填写的文本框。它有一个运行模式,您可以在其中将测试添加到套件中,然后运行该套件。Lisa 和 Janet 都没有尝试过这个工具,但是在 Yahoo 敏捷测试组中,它被认为是一个用于测试 Web 服务的工具,如果你每次都运行相同的数据的话。
CrossCheck. CrossCheck is one example of a tool for testing web services. You supply the WSDL (Web Services Description Language); CrossCheck compiles the page and then presents you with a tabbed menu that contains textboxes for you to fill in. It has a Run mode where you can add your tests to a suite and then run the suite. Neither Lisa or Janet have tried this tool, but it was noted on the Yahoo agile-testing group as a tool to use for testing web services if you were running the same data through each time.
Ruby 测试::单元。Janet 参与的一个项目使用 Ruby 的单元测试框架 Test::Unit 来测试 Web 服务,并取得了巨大成功。事实上,该团队能够尽早进行测试,为程序员提供即时反馈,这有助于最终设计。
Ruby Test::Unit. One project Janet was on used Ruby’s unit testing framework, Test::Unit, to test web services, with great success. In fact, the team was able to test early to give the programmers immediate feedback, which helped with the final design.
请参阅第 12 章“测试象限总结”中的“系统测试”示例,了解 Janet 的团队如何使用 Ruby Test::Unit 来测试 Web 服务。
See the “System Test” example in Chapter 12, “Summary of Testing Quadrants,” to see how Janet’s team used Ruby Test::Unit to test web services.
肥皂用户界面。建议用于测试 Web 服务的另一个工具是 soapUI。它具有陡峭的学习曲线,但可用于性能和负载测试。因为它可以遍历 Excel 电子表格或文本文件中的行,所以它可以用于数据驱动的测试。
soapUI.Another tool suggested for testing web services is soapUI. It has a steep learning curve but can be used for performance and load testing. Because it can loop though rows in an Excel spreadsheet or text file, it can be used for data-driven testing.
在表示层以下的层工作的测试非常适合编写和自动化指导编码的客户测试。一些从业者还没有从故事测试驱动开发中获得他们期望的价值。Brian Marick [ 2008 ] 假设使用程序员测试驱动开发构建的应用程序、大量示例的面向业务的设计严重依赖于白板讨论、一小组自动完整性测试和大量探索性测试可能更便宜且同样有效的方法。无论采用哪种方法,如果您要测试带有用户界面的应用程序,您都需要在 GUI 级别实现一些自动化。
Tests that work at the layers below the presentation layer are well suited for writing and automating customer tests that guide coding. Some practitioners haven’t gotten the value they expected from story test-driven development. Brian Marick [2008] hypothesized that an application built with programmer test-driven development, example-heavy business-facing design that relies heavily on whiteboard discussions, a small set of automated sanity tests, and lots of exploratory testing could be a less expensive and equally effective approach. Whichever approach you take, if you’re testing an application with a user interface, you’ll need some automation at the GUI level.
等一下。我们如何使用 GUI 测试来驱动开发,因为 GUI 在故事完成之前不会准备好?这听起来有悖常理,但自动化 GUI 测试对我们开发新功能非常重要。测试框架可用于在编写代码之前为 GUI 工具指定测试用例。此外,您可以在之前自动化 GUI 测试通过使用 HTML 模型或通过所有屏幕开发一个端到端的准系统切片来完成编码,这些屏幕只是简单地导航但尚未提供所有功能。即使您没有使用大量自动化故事测试来推动开发,但如果没有自动化的任何帮助,帮助我们了解功能并提供即时反馈的手动探索性测试也会变得非常乏味和缓慢。让我们看看有助于使用面向业务的测试推动开发的 GUI 测试工具的类型。
Wait a minute. How can we use GUI tests to drive development, because the GUI won’t be ready until the story is complete? It sound counterintuitive, but automated GUI tests are important to help us while we’re developing new functionality. Test frameworks can be used to specify test cases for a GUI tool before the code is written. In addition, you can automate GUI tests before coding is finished, either by using HTML mock-ups or by developing an end-to-end bare-bones slice through all of the screens that simply navigates but doesn’t provide all of the functionality yet. Even if you’re not using a lot of automated story tests to drive development, manual exploratory testing that helps us learn about the functionality and provides immediate feedback gets pretty tedious and slow without any assistance from automation. Let’s look at the types of GUI test tools that help drive development using business-facing tests.
录制/回放工具很有吸引力,因为您通常可以学会如何录制脚本并快速回放,并且可以在短时间内创建大量脚本。然而,它们也有缺点。早期的 GUI 测试工具使用 XY 屏幕坐标记录鼠标移动。使用这些工具的脚本也可能对屏幕分辨率、颜色深度甚至窗口在屏幕上的放置位置的变化很敏感。
Record/playback tools are appealing because you can usually learn how to record a script and play it back quickly, and you can create lots of scripts in a short time. However, they have drawbacks. Early GUI test tools recorded mouse movements using X-Y screen coordinates. Scripts using those tools might also be sensitive to changes in screen resolution, color depth, and even where the window is placed on the screen.
大多数现代 GUI 测试工具都使用对象来识别图形应用程序中的控件,例如按钮、菜单和文本输入小部件,因此它们可以通过符号而不是原始屏幕坐标来引用它们。这使得应用程序更易于测试,因为它更强大地经受住变化。一个按钮可能会移动到屏幕的不同部分,但测试仍然可以根据它的对象名称找到它。
Most modern GUI test tools use objects to recognize the controls in a graphical application, like buttons, menus, and text input widgets, so they can refer to them symbolically rather than with raw screen coordinates. This makes the application much more testable, because it’s more robust standing up to changes. A button might move to a different part of the screen, but the test can still find it based on its object name.
即使改进了对象识别,使用记录/回放创建的脚本通常也很脆弱且维护成本高。录制可能是开始创建脚本的好方法。了解工具脚本语言的测试人员或程序员可以将录制的脚本重构为更易于使用和维护的面向对象模型。从历史上看,记录/回放工具使用专有的脚本语言,程序员对学习这些语言不感兴趣。更改测试中使用的设计模式也更加困难。
Even with improved object recognition, scripts created with record/playback are usually brittle and expensive to maintain. Recording can be a good way to start creating a script. Testers or programmers who know the tool’s scripting language can refactor the recorded script into an object-oriented model that’s easier to use and maintain. Historically, record/playback tools used proprietary scripting languages, which programmers aren’t interested in learning. It’s also more difficult to change the design patterns used in the tests.
一些基于脚本的工具(例如我们将在接下来几节中讨论的工具)提供记录功能,以帮助人们快速开始编写测试脚本。但是,使用这些工具,录制的脚本并不是为了直接播放而设计的;它们只是创建设计良好且易于维护的测试套件的起点。
Some script-based tools such as the ones we’ll talk about in the next few sections offer a record feature to help people get a quick start on writing the test script. However, with those tools, the recorded scripts aren’t intended for straight playback; they’re just a starting point to creating a well-designed and easily maintained suite of tests.
许多敏捷团队更喜欢可以让他们创建自己的领域特定语言 (DSL) 的工具和脚本语言。这使得业务专家更容易理解甚至编写测试。接下来让我们看看其中的一些。
Many agile teams prefer tools and scripting languages that let them create their own domain-specific language (DSL). This makes tests much easier for business experts to understand and even write. Let’s look at some of these next.
本节中的每个工具最初都是由一个敏捷开发团队编写的,该团队需要一个 GUI 测试工具并且找不到任何适合其情况的第三方工具。使用这些工具,您可以编写像人类用户一样使用 Web 应用程序的脚本。他们填写文本字段,从列表中进行选择,然后单击复选框和按钮。它们提供了多种方法来验证页面的正确导航和内容,例如特定于工具的验证步骤或 XPath。其中一些工具的学习曲线比简单的录制/回放工具更高,但额外的时间投资通常会在总拥有成本较低的脚本中得到回报。
Each of the tools in this section was originally written by an agile development team that needed a GUI test tool and couldn’t find any third-party tools that worked for its situation. With these tools, you can write scripts that use web applications just like a human user. They fill in text fields, select from lists, and click checkboxes and buttons. They provide a variety of ways to verify correct navigation and contents of pages, such as tool-specific verify steps or XPath. Some of these tools have a higher learning curve than simple record/playback tools, but the extra investment of time usually pays off in scripts with a low total cost of ownership.
Ruby 与 Watir。Watir(Ruby 中的 Web 应用程序测试)是一个简单的开源 Ruby 库,用于自动化与 Windows 上的 Internet Explorer 配合使用的 Web 浏览器。其他浏览器有不同的风格,包括用于 Firefox 的 FireWatir 和用于 Safari 的 SafariWatir。
Ruby with Watir. Watir (Web Application Testing in Ruby) is a simple open source Ruby library for automating web browsers that works with Internet Explorer on Windows. There are different flavors for other browsers, including FireWatir for Firefox and SafariWatir for Safari.
您使用的任何工具总是有缺点。例如,使用对象有限制。有时,程序员会使用您的工具可能无法理解的自定义控件或新工具包。
There are always drawbacks to any tool you use. For example, there are limitations to using objects. Sometimes programmers use custom controls or a new toolkit that your tool might not understand.
实施新的测试自动化工具通常需要进行一些实验,以在可测试代码和设计良好的测试脚本之间取得良好的平衡。让整个团队参与进来会让这件事变得容易得多。Watir 是我们发现非常适合敏捷项目的 GUI 测试工具的一个示例。让我们再看几个,Selenium 和 Canoo WebTest。
Implementing a new test automation tool usually requires some experimentation to get a good balance of testable code and well-designed test scripts. Involving the whole team makes this much easier. Watir is one example of a GUI test tool that we’ve found is well suited to agile projects. Let’s look at a couple more, Selenium and Canoo WebTest.
硒。Selenium 是另一个开源工具,实际上是一套工具,用于测试 Web 应用程序。测试可以编写为 HTML 表格或用多种流行的编程语言编码,并且可以直接在大多数现代网络浏览器中运行。名为“Selenium IDE”的 Firefox 插件提供了一种快速学习该工具的方法。提供了一个记录器来帮助创建测试,包括编写断言。可以使用多种不同的通用编程和脚本语言编写测试,包括 Java、C# 和 Ruby。
Selenium. Selenium is another open source tool, actually a suite of tools, for testing web applications. The tests can be written as HTML tables or coded in a number of popular programming languages, and can be run directly in most modern web browsers. A Firefox plug-in called “Selenium IDE” provides a way to learn the tool quickly. A recorder is provided to help create the tests, including writing assertions. Tests can be written in several different common programming and scripting languages, including Java, C#, and Ruby.
有关使用 Selenium RC 创建特定领域的测试自动化框架的示例,请参阅第 14 章,“敏捷测试自动化策略”。
See Chapter 14, “An Agile Test Automation Strategy,” for an example of using Selenium RC to create a domain-specific test automation framework.
Canoo 网络测试。在 WebTest 脚本中,测试被指定为 XML 文件中的“步骤”,通过 Web UI 模拟用户的操作。下面是 WebTest 脚本如何调用页面并验证结果的示例:
Canoo WebTest. In WebTest scripts, tests are specified as “steps” in XML files, simulating a user’s actions through a web UI. Here’s an example of how a WebTest script might invoke a page and verify the results:
<setInputField description="set query" name="q" value="Agile Tester"/>
<clickButton description="submit query" label="Google Search"/>
<verifyText description="check for result" text="Lisa Crispin" />
<verifyText description="check for result" text="Janet Gregory" />
<setInputField description="set query" name="q" value="Agile Tester"/>
<clickButton description="submit query" label="Google Search"/>
<verifyText description="check for result" text="Lisa Crispin" />
<verifyText description="check for result" text="Janet Gregory" />
WebTest 不像 Selenium 和 Watir 那样驱动实际的浏览器,而是使用 HtmlUnit 模拟所需的浏览器。与编写测试脚本相比,指定测试的优势在于它们没有逻辑,您不必测试测试。
Rather than driving an actual browser, as Selenium and Watir do, WebTest simulates the desired browser using HtmlUnit. The advantage of specifying tests as opposed to coding test scripts, is because there’s no logic in them, you don’t have to test the test.
在我们撰写本书时,Selenium、WebTest 和 Watir 只是众多可用于 GUI 测试的开源工具中的三个示例。许多团队编写自己的测试自动化框架。让我们在下一节中看一个示例。
Selenium, WebTest, and Watir are just three examples of the many open source tools available for GUI testing as of the time we wrote this book. Many teams write their own test automation frameworks. Let’s look at an example in the next section.
Bret Pettichord [ 2004 ] 创造了术语“自制”来指代敏捷团队为满足他们自己独特的测试需求而创建的工具。这允许比开源工具更多的定制。这些工具的目标通常是为非技术客户团队成员和测试人员提供一种方式来编写实际上可由自动化工具执行的测试。自制工具是根据项目的确切需求量身定制的。它们的设计可以最大限度地降低总拥有成本。它们通常建立在现有开源工具之上。
Bret Pettichord [2004] coined the term “home-brewed” for the tools agile teams create to meet their own unique testing needs. This allows even more customization than an open source tool. The goal of these tools is usually to provide a way for nontechnical customer team members and testers to write tests that are actually executable by the automated tool. Home-brewed tools are tailored to the exact needs of the project. They can be designed to minimize the total cost of ownership. They’re often built on top of existing open source tools.
Janet 参与了一些使用 Ruby 和 Watir 创建功能测试完整框架的项目。这些框架允许客户指定测试,然后将其转化为功能回归套件。
Janet has been involved in a few projects that have used Ruby and Watir to create a full framework for functional testing. These frameworks allowed customers to specify tests that were then turned into a functional regression suite.
没有测试工具可以保证成功。事实上,测试自动化的历史充满了失败的尝试。让整个团队考虑使用最好的工具是一个很大的帮助,但无论你使用什么工具,你都需要一种聪明的方法来编写测试。我们将在下一节中讨论。
No test tool guarantees success. In fact, the history of test automation is littered with failed attempts. Having the whole team think about the best tools to use is a big help, but no matter what tool you use, you need a smart approach to writing tests. We’ll discuss that in the next section.
如果您不明智地使用它们,那么世界上最好的工具也无济于事。测试工具可能使指定测试变得非常容易,但是您是否在正确的时间指定正确的测试取决于您。Lisa 的团队发现,过多的前期细节会使大局蒙上阴影,以至于程序员不知道要编写什么代码。并非每个团队都如此,在某些时候我们确实需要详细信息。提供它们的最晚时间是当程序员拿起编码任务卡并开始编写故事时。
The best tools in the world won’t help if you don’t use them wisely. Test tools might make it very easy to specify tests, but whether you’re specifying the right tests at the right time is up to you. Lisa’s team found that too much detail up front clouded the big picture to such a degree that the programmers didn’t know what to code. This won’t be true for every team, and at some point we do need details. The latest time to provide them is when a programmer picks up a coding task card and starts working on a story.
编写能够有效传达所需行为的详细测试用例需要艺术和科学。表达不当的场景和设计不当的测试用例会造成比解决问题更多的混乱。进行实验,以便您可以为每个故事找到正确的详细程度和正确的测试设计。让我们看看一些策略,以帮助您成功地使用工具来编写有用的面向业务的测试。
Writing detailed test cases that communicate desired behavior effectively requires both art and science. Poorly expressed scenarios and poorly designed test cases can create more confusion than they resolve. Experiment so that you can find the right level of detail and the right test design for each story. Let’s look at some strategies to help you use tools successfully to write useful business-facing tests.
在我们定义了高级验收测试以便程序员知道开始编码的内容之后,我们可以开始详细说明故事测试的其余部分。我们可以与程序员密切合作,以确保我们以最佳方式实现自动化。
After we have defined our high-level acceptance tests so that the programmer knows what to start coding, we can start elaborating on the rest of the story tests. We can work closely with the programmer to ensure we automate the best possible way.
当程序员开始处理故事的编程任务时,请开始编写详细的测试。对于我们这些喜欢测试的人来说,很容易立即去寻找最大的“气味”,即我们认为代码可能脆弱的区域。抵制诱惑。确保最明显的用例首先起作用。编写一个简单、快乐的路径自动化测试,以显示代码完成了它应该完成的最基本的任务。测试通过后,您就可以开始变得更有创意了。编写面向业务的测试是一个迭代过程。
When a programmer starts working on the programming tasks for a story, start writing detailed tests. For those of us who enjoy testing, it’s tempting to go for the biggest “smells” right away, the areas where we think the code might be fragile. Resist the temptation. Make sure the most obvious use case is working first. Write a simple, happy path automated test to show the code accomplishes the most basic task that it should. After that test passes, you can start getting more creative. Writing the business-facing tests is an iterative process.
第 18 章,“编码和测试”,详细介绍了测试人员和程序员如何协同工作以进行测试和编码。
Chapter 18, “Coding and Testing,” goes into more detail about how testers and programmers work together to test and code.
将每个测试限制在一个业务规则或条件中。在某些时候,您可以自动或手动执行更复杂的场景,但首先要用一个简单的测试来涵盖每个条件。如果您遵循了我们推荐的薄片或钢丝模式,第一组测试应该证明第一端到端的薄片。当您的自动化测试通过时,将它们添加到在频繁构建过程中运行的回归套件中。
Confine each test to one business rule or condition. At some point you can automate or manually perform more complex scenarios, but start by covering each condition with a simple test. If you’ve followed our recommended thin slice or steel thread pattern, the first set of tests should prove the first thin slice end-to-end. As your automated tests pass, add them to the regression suite that runs in a frequent build process.
测试通过后,除非需求发生变化,否则不应失败。如果发生这种情况,则应在更改代码之前更新测试。当然,如果作为需求变更的一部分忘记了测试,我们预计它会失败。它完成了它作为变化检测器的工作。此时,测试可能需要更改才能通过。
After a test passes, it shouldn’t fail unless the requirements were changed. If that happens, the test should be updated before the code is altered. Of course, if a test was forgotten as part of a requirement change, we expect it to fail. It did its job as change detector. At this time, the test will likely need to change to get it passing.
每当持续集成和构建过程中的测试失败时,团队的最高优先级(关键生产问题除外)应该是让构建再次通过。不要注释掉失败的测试并在以后修复它;那是灭亡之路。很快你就会有几十个注释掉的测试和大量的技术债务。团队中的每个人都应该停止他们正在做的事情并确保构建再次“绿色”。确定是否引入了错误,或者是否只需要更新测试以适应有意更改的行为。修复问题,将其签入,并确保所有测试都通过。
Whenever a test fails in a continuous integration and build process, the team’s highest priority (other than a critical production problem) should be to get the build passing again. Don’t comment out the failing test and fix it later; that’s the road to perdition. Soon you’ll have dozens of commented-out tests and a lot of technical debt. Everyone on the team should stop what they’re doing and make sure the build goes “green” again. Determine if a bug has been introduced, or if the test simply needs to be updated to accommodate intentionally changed behavior. Fix the problem, check it in, and make sure all of the tests pass.
一个通过的测试会导致另一个。通过重构使您的测试保持最新和可维护。扩展它们以涵盖其他测试用例。各种组合和场景在通过后可能会或可能不会成为回归套件的一部分。我们希望我们的回归套件能够及时运行,对边缘情况进行过多测试会减慢速度。
One passing test leads to another. Keep your tests current and maintainable with refactoring. Extend them to cover other test cases. The various combinations and scenarios might or might not become part of the regression suite after they pass. We want our regression suite to run in a timely manner, and having too many tests for edge cases would slow it down.
在设计测试时,查看不同的模式并选择适合您的模式。使它们尽可能简单。在设计测试之前,您必须确定所需的测试。Pierre Veragen 创造了术语测试起源模式来表示帮助您思考测试的模式。示例和用例提供给我们的测试起源模式。
When designing tests, look at different patterns and choose the ones that work for you. Keep them as simple as you can. Before you can design tests, you have to identify the ones you need. Pierre Veragen coined the term test genesis patterns to note the patterns that help you think of tests. Examples and use cases feed into our test genesis patterns.
Lisa 的团队通常采用构建/操作/检查模式:根据测试目的,在内存中或实际上在数据库中构建输入数据;调用生产代码对这些输入进行操作;并检查该操作的结果。一些团队将此称为设置/执行/验证。例如,要测试提供给新账户持有人的发票,设置要收取的费用,输入与费用金额相关的账户属性,运行计算费用的代码,然后检查费用是多少实际收费。见图9-9举一个测试示例,该测试设置了具有指定金额、利率、期限、付款频率和服务开始日期的贷款,然后检查生成的分期付款时间表。测试数据内置于内存中,有利于快速测试。“拆卸”夹具(未显示)从内存中删除测试数据,因此不会干扰后续测试。
Lisa’s team often goes with a build/operate/check pattern: Build the input data, in memory or actually in the database, depending on the purpose of the test; invoke the production code to operate on those inputs; and check the results of that operation. Some teams call this setup/execute/validate. For example, to test the invoice presented to a new account holder, set up the fees to be charged, input the properties of the account that relate to fee amounts, run the code that calculates the fees, and then check to see what fees were actually charged. See Figure 9-9 for an example of a test that sets up a loan with a specified amount, interest rate, term, payment frequency, and service start date and then checks the resulting amortization schedule. The test data is built in memory, which makes for a speedy test. A “teardown” fixture (not shown) removes the test data from memory so it won’t interfere with subsequent tests.
如果需要测试应用程序的数据访问层,可以使用实际数据库运行测试。每个测试都可以插入自己需要的测试数据,进行操作,检查结果,删除数据。使用真实数据库中的数据进行测试可以成为针对数据访问层和业务逻辑层不易分离的遗留代码进行自动化测试的一种方法。
If there’s a need to test the application’s data access layer, tests can run using an actual database. Each test can insert the test data it needs, operate on it, check results, and delete the data. Testing with data in a real database can be a means of automating a test against legacy code whose data access and business logic layers aren’t easily separated.
请注意,示例中的“check”表使用声明式样式,每一行构成一个独立的测试用例,不会改变系统的状态。我们示例中的每一行都测试贷款摊销计划中的一行。在下一节中,我们将研究过程风格的模式,其中包含更改或测试系统状态的步骤。
Notice that the “check” table in the example uses a declarative style, with each row forming an independent test case, without changing the state of the system. Each row in our example tests a line in the loan amortization schedule. In the next section, we’ll look at patterns that are in a procedural style, with steps that change or test the state of the system.
有时,基于时间线的程序模式可以更好地反映业务。例如,在测试贷款时,我们希望确保每笔付款都正确应用了利息和本金。利息金额取决于收到付款的日期和最后处理付款的日期。我们想要一个模拟以一定的金额、利率和时间段贷款的测试,然后随着时间的推移模拟借款人发送付款,这些付款会被接收和处理。图 9-10显示了 FitLibrary“DoFixture”测试的一个简单示例,该测试提取贷款、检查付款金额、发布借款人的付款、接收付款并处理它们,然后检查利息、本金和贷款余额. 它还检查贷款违约状态。
Sometimes a timeline-based procedural pattern reflects the business better. For example, when testing a loan, we want to make sure interest and principal are applied correctly for each payment. The amount of interest depends on the date the payment was received and the date of the last payment processed. We want a test that simulates taking out a loan for a certain dollar amount, interest rate, and time period, and then over time simulates the borrower sending in payments, which are received and processed. Figure 9-10 shows a simple example of a FitLibrary “DoFixture” test that takes out a loan, checks the payment amount, posts the borrower’s payments, receives the payments and processes them, and then checks the interest, principal, and loan balance amount. It also checks the loan default state.
根据领域的不同,基于时间或事件的方法可能比声明式类型测试更好地模拟实际业务流程并且更容易被业务专家理解。其他客户可能会发现声明式表格样式更易于理解,因为它隐藏了程序细节。不同的模式最适合不同的情况,因此请尝试使用它们。
Depending on the domain, a time- or event-based approach might simulate the actual business processes better and be more understandable to business experts than a declarative type test. Other customers might find the declarative table style simpler to understand, because it hides the procedural details. Different patterns work best for different situations, so experiment with them.
您的团队应该自学有助于推动编程的测试模式。为每种类型的测试找到正确的模式可确保测试传达清晰、易于维护并在最佳时间内运行。有关测试设计的更多宝贵资源,请参阅参考书目,例如 Gerard Meszaros 的xUnit 测试模式:重构测试代码。
Your team should educate itself on test patterns that help drive programming. Finding the right pattern for each type of test ensures the test communicates clearly, is easy to maintain, and runs in an optimal amount of time. See the bibliography for more invaluable resources on test design, such as Gerard Meszaros’s xUnit Test Patterns: Refactoring Test Code.
将程序员和测试人员聚集在一起,集思广益测试方法,并帮助决定哪些测试可以自动化以及代码应该如何设计以支持测试。业务逻辑和算法应该可以通过测试装置访问,而无需通过用户界面或批处理调度过程。这使得测试驱动开发成为可能,进而产生可测试的架构。
Bring programmers and testers together to brainstorm test approaches and to help decide what tests can be automated and how the code should be designed to support testing. Business logic and algorithms should be accessible by test fixtures, without having to go through a user interface or batch scheduling process. This enables test-driven development, which in turn produces testable architecture.
自动化测试的一种常见方法是使用关键字或动作词来驱动测试。这可以与 Fit 和 FitNesse 等工具一起使用,或者与 Watir 一起使用 Ruby。我们接下来会解释这个。
A common approach to automating tests is by driving tests with keywords or action words. This can be used with tools such as Fit and FitNesse, or Ruby with Watir. We’ll explain this next.
数据驱动测试是一种有助于减少测试维护并使您能够与手动测试人员共享测试自动化的工具。很多时候您想要一遍又一遍地运行相同的测试代码,只重复输入和预期结果。电子表格或表格(例如 Fit 支持的表格)是指定输入的绝佳方式。测试夹具、方法或脚本可以一次循环遍历每个数据值,将预期结果与实际结果相匹配。通过使用数据驱动测试,您实际上是在使用示例来展示应用程序应该做什么。
Data-driven testing is a tool that can help reduce test maintenance and enable you to share your test automation with manual testers. There are many times when you want to run the same test code over and over, repeating only the inputs and expected results. Spreadsheets or tables, such as those supported by Fit, are excellent ways to specify inputs. The test fixture, method, or script can loop through each data value one at a time, matching expected results to actual results. By using data-driven tests, you are actually using examples to show what the application is supposed to do.
关键字驱动测试是自动化测试中使用的另一种工具,其中使用预定义的关键字来定义操作。这些动作对应于与应用程序相关的进程。这是创建域测试语言的第一步。这些关键字(或动作词)代表一种非程序员可以用来开发自动化测试的非常简单的规范语言。您仍然需要程序员或技术自动化专家来实现动作词所作用的固定装置。如果扩展这些关键字以模拟领域语言,客户和非技术测试人员可以更轻松地指定映射到工作流的测试。
Keyword-driven testing is another tool used in automated testing, where predefined keywords are used to define actions. These actions correspond to a process related to the application. It is the first step in creating a domain testing language. These keywords (or action words) represent a very simple specification language that non-programmers can use to develop automated tests. You still need programmers or technical automation specialists to implement the fixtures that the action words act on. If these keywords are extended to emulate the domain language, customers and nontechnical testers can specify tests that map to the workflow more easily.
图 9-11中的示例电子表格显示了一家公司如何使用动作词来自动化他们的测试设置。可以使用相同的动作词进行测试。Signup、Signoff 和 CCDeposit 是特定于域的词。他们的用户可以在不了解底层代码的情况下轻松编写测试。
The sample spreadsheet in Figure 9-11 shows how one company used action words to automate their test setup. The same action words can be used to test. The words Signup, Signoff, and CCDeposit are words that are domain-specific. Their users could easily write tests without understanding the underlying code.
结合数据驱动和关键字驱动的测试技术会非常强大。Fit 和 FitNesse 同时使用关键字和数据来驱动测试。我们在本章中描述的其他工具也可以采用这种方法。
Combining data-driven and keyword-driven testing techniques can be very powerful. Fit and FitNesse use both keywords and data to drive tests. The other tools we’ve described in this chapter can also accommodate this approach.
如果代码不是为易于测试而设计的,那么任何测试策略都可能会遇到麻烦。让我们来看看可测试性问题。
Any test strategy can run into trouble if the code isn’t designed to be easily tested. Let’s take a look at testability concerns.
使用适当的设计模式构建并在任何编码之前编写的面向业务的测试可帮助团队实现可测试的代码设计。程序员从查看面向业务的测试开始,也许与测试人员、分析师或客户一起,以便在他们继续进行测试驱动设计时,执行这些测试的需求始终在他们的脑海中。他们可以构建,以便测试提供输入并控制运行时条件。
Business-facing tests built with appropriate design patterns and written ahead of any coding help the team achieve a testable code design. The programmers start by looking at the business-facing tests, perhaps together with a tester, analyst, or customer, so that the need to execute those tests are always in their minds as they proceed with their test-driven design. They can build so that the tests provide inputs and control run-time conditions.
让我们看看促进可测试代码设计的技术。
Let’s look at techniques that promote design of testable code.
在第 7 章“支持团队的面向技术的测试”中,我们解释了单元级别的测试驱动开发如何确保可测试的架构。面向业务的测试也是如此。Lisa 团队设计的分层架构同样适用于功能测试。可以直接针对业务逻辑进行测试,而无需涉及用户界面,如果合适,也无需涉及数据库层。这并不意味着不需要测试数据库层。它仍然需要测试,也许在其他地方。
In Chapter 7, “Technology-Facing Tests that Support the Team,” we explained how test-driven development at the unit level ensures a testable architecture. This is true for business-facing tests as well. The layered architecture Lisa’s team designed works just as well for functional testing. Testing can be done directly against the business logic without involving the user interface, and if appropriate, without involving the database layer. This doesn’t mean that the database layer doesn’t need to be tested. It still needs to be tested, just maybe somewhere else.
在对表示层进行编码时也必须考虑可测试性。GUI 测试工具在使用良好实践开发的设计良好的代码上工作得更好。
Testability has to be considered when coding the presentation layer as well. GUI test tools work better on well-designed code developed with good practices.
编码和测试是敏捷开发过程中的一部分。代码设计和测试设计是相辅相成、相互依存的。这是一个先有鸡还是先有蛋的场景:没有可测试的代码设计就无法编写测试,而没有设计良好的测试(明确传达需求并与系统架构兼容)就无法编写代码。这就是为什么我们总是将编码和测试放在一起考虑。当我们估算故事时,我们包括编码和测试的时间,当我们计划每个迭代和故事时,我们预算时间来设计测试和代码。如果证明自动化测试很困难,请评估代码设计。如果程序员编写的代码不符合客户的期望,问题可能出在测试设计不当。
Coding and testing are part of one process in agile development. Code design and test design are complementary and interdependent. It’s a chicken-andegg scenario: You can’t write tests without a testable code design, and you can’t write code without well-designed tests that clearly communicate requirements and are compatible with the system architecture. This is why we always consider coding and testing together. When we estimate stories, we include time for both coding and testing, and when we plan each iteration and story, we budget time to design both tests and code. If automating a test proves difficult, evaluate the code design. If programmers are writing code that doesn’t match customer expectations, the problem might be poorly designed tests.
我们假设至少有相当一部分指导编程的测试是自动化的。如果您及早与程序员共享手动测试场景,它们也可以推动编程。越早将它们转化为自动化测试,您就会越快地实现收益。大多数手动测试更多地属于“评论产品”象限,在那里我们可能会了解我们在最初的一组测试中没有预料到的故事。
We’ve assumed that at least a good-sized portion of the tests that guide programming will be automated. Manual test scenarios can also drive programming if you share them with the programmers early. The earlier you turn them into automated tests, the faster you will realize the benefit. Most manual tests fall more into the “critique product” quadrant where we might learn things about the story we hadn’t anticipated with the initial set of tests.
在第 IV 部分“测试自动化”中,我们将深入探讨如何开发成功的测试自动化策略,并研究诸如构建您自己的工具与使用第三方或开源工具之类的考虑因素。
In Part IV, “Test Automation,” we’ll dive into developing a successful test automation strategy and look at considerations such as building your own tools versus using third-party or open source tools.
这并不能阻止我们编写可能不适合自动化的测试。编写测试时不要担心细节。您可能会想出一次性测试,这些测试很重要,但在回归套件中一遍又一遍地重复并不重要。您可能会开始考虑端到端的场景或探索性测试会话的跳板,这些会话可能会通过一些自动化来促进,但需要一个聪明的人来全面执行它们。你稍后会明白的。现在,我们要确保我们捕捉到了客户的关键需求。
That doesn’t stop us from writing tests that might not be appropriate for automation. Don’t sweat the details when you’re writing tests. You might come up with one-off tests that are important to do but not important to repeat over and over in a regression suite. You might start thinking about end-to-end scenarios or springboards to exploratory test sessions that might be facilitated with some automation but need an intelligent human to conduct them in full. You’ll figure that out later. Right now, we want to make sure we capture the customer’s critical requirements.
从一个简单的方法开始,看看它是如何工作的,并以此为基础进行构建。重要的是开始编写面向业务的测试以在开发产品时支持团队。
Start with a simple approach, see how it works, and build on it. The important thing is to get going writing business-facing tests to support the team as you develop your product.
如果我们要自动化测试,将它们呈现在自动化工具框架中是有意义的,即使它们还不可执行。我们希望所有测试(即使是那些不会自动进行的测试)都可以为开发团队中的每个人访问并为我们的客户所理解。有很多选项可以让团队中的每个人都看到测试。Wiki 是共享测试用例的常用方式,一些工具(如 FitNesse)使用 wiki 或类似工具,使叙述性需求、示例和可执行测试共存于一个地方。
If we’re automating tests, it makes sense to present them in the automation tool framework, even if they’re not yet executable. We want some way for all tests, even those that won’t be automated, to be accessible to everyone on the development team and understandable to our customers. There are lots of options available that let everyone on the team see tests. Wikis are a common way to share test cases, and some tools such as FitNesse use a wiki or similar tool, enabling narrative requirements, examples, and executable tests to coexist in one place.
第 14 章,“敏捷测试自动化策略”,详细介绍了如何管理自动化测试。
Chapter 14, “An Agile Test Automation Strategy,” goes into more detail on how to manage automated tests.
测试应包含在您的源代码控制中,以便您可以跟踪哪些版本的测试与哪些版本的代码一致。至少,对您的测试进行某种版本控制。一些团队使用可能与需求管理、缺陷跟踪或其他组件集成的测试管理工具或综合测试框架。
Tests should be included in your source code control, so that you can track which versions of the tests go with which versions of the code. At the very least, have some kind of version control for your tests. Some teams use test management tools or comprehensive test frameworks that might integrate with requirements management, defect tracking, or other components.
在本章中,我们了解了您可能希望在工具包中使用的工具来帮助创建面向业务的测试,以帮助推动开发和指南,以确保这些工具提供帮助而不是阻碍。这些工具和指南包括以下内容:
In this chapter, we’ve looked at tools you might want in your toolkit to help create business-facing tests that help drive development and guidelines to make sure the tools help rather than get in the way. The tools and guidelines included the following:
团队需要正确的工具来引出需求和示例,从全局到细节,包括清单、思维导图、电子表格、模型、流程图和各种基于软件的工具。
Teams need the right tools to elicit requirements and examples, from the big picture down to details, including checklists, mind maps, spreadsheets, mock-ups, flow diagrams, and various software-based tools.
在 GUI 下方和通过 GUI 表达示例和自动化测试的工具对于敏捷测试自动化也是必不可少的。其中一些工具包括单元测试工具、行为驱动开发工具、FitNesse、Ruby with Watir、Selenium 和 Canoo WebTest。
Tools to express examples and automate tests, below and through the GUI, are also essential to agile test automation. Some of these tools include unit test tools, behavior-driven development tools, FitNesse, Ruby with Watir, Selenium, and Canoo WebTest.
“自制”测试自动化可帮助团队将自动化测试的总拥有成本保持在较低水平。
“Home brewed” test automation helps teams keep the total cost of ownership of their automated tests low.
通过面向业务的测试推动开发是激励敏捷团队设计可测试代码的一种方式。
Driving development with business-facing tests is one way agile teams are motivated to design testable code.
构建自动化的测试策略应该包括逐步构建测试并确保它们始终通过。设计模式可用于帮助您创建有效的测试。
Test strategies for building your automation should include building your tests incrementally and making sure they always pass. Design patterns can be used to help you create effective tests.
关键字和数据驱动的测试是与我们在本章中讨论的工具一起使用的常用方法。
Keyword and data-driven testing is a common approach that works with the tools we’ve discussed in this chapter.
在您的代码设计中考虑可测试性,并明智地选择您的测试工具,因为它们需要与您的代码一起工作。
Consider testability in your code design, and choose your test tools wisely, because they need to work with your code.
我们需要一些方法来组织测试,以便可以有效地使用它们并将其置于版本控制中。
We need some way to organize tests so that they can be used effectively and put into version control.
本章涵盖测试矩阵的第三象限。在第 8 章“支持团队的面向业务的测试”中,我们讨论了第二象限以及如何使用面向业务的测试来支持编程。在本章中,我们将向您展示如何通过不同类型的面向业务的测试来评价产品。我们还将讨论可能有助于这些活动的工具。
This chapter covers the third quadrant of the testing matrix. In Chapter 8, “Business-Facing Tests that Support the Team,” we talked about the second quadrant and how to use business-facing tests to support programming. In this chapter, we show you how to critique the product with different types of business-facing tests. We’ll also talk about tools that might help with these activities.
请记住,面向业务的测试是您可以用业务专家会(或应该)感兴趣的术语来描述的测试。当我们提到传统分阶段方法中的测试时,它几乎总是意味着在产品构建后对其进行评判。到目前为止,您可能认为在敏捷开发中这部分测试应该很容易。毕竟,我们只是花了所有时间来确保它按预期工作。这些需求在构建时都经过了测试,包括安全性和其他非功能性需求,对吧?剩下的就是可能找到一些晦涩或有趣的错误。
Remember that business-facing tests are those you could describe in terms that would (or should) be of interest to a business expert. When we mention testing in traditional phased approaches, it pretty much always means critiquing the product after it is built. By now, you might think that in agile development this part of testing should be easy. After all, we just spent all that time making sure it works as expected. The requirements have all been tested as they were built, including security and other nonfunctional requirements, right? All that’s left is to possibly find some obscure or interesting bugs.
作为测试人员,我们知道人们会犯错误。无论我们多么努力地在第一次做对,我们有时还是会做错。也许我们使用了一个没有测试我们认为它做了什么的例子。或者,也许我们记录了一个错误的预期结果,所以测试通过了,但它是一个误报。业务专家可能忘记了真实用户需要的一些东西。最好的客户在看到之前可能不知道自己想要(或不想要)什么。
As testers, we know that people make mistakes. No matter how hard we try to get it right the first time, we sometimes get it wrong. Maybe we used an example that didn’t test what we thought it did. Or maybe we recorded a wrong expected result so the test passed, but it was a false positive. The business expert might have forgotten some things that real users needed. The best customer may not know what she wants (or doesn’t want) until she sees it.
批评或评估产品是测试人员或业务用户在评估和判断产品时所做的事情。这些评估者根据他们是否喜欢新屏幕的行为方式、外观和感觉或工作流程来形成看法。与想象产品向您描述时的样子相比,看到、感觉和触摸产品并做出反应要容易得多。
Critiquing or evaluating the product is what testers or business users do when they assess and make judgments about the product. These evaluators form perceptions based on whether they like the way it behaves, the look and feel, or the workflow of new screens. It is easier to see, feel, and touch a product and respond than to imagine what it will look like when it is described to you.
很难将批评产品的面向业务的测试自动化,因为此类测试依赖于人类的智慧、经验和直觉。然而,自动化工具可以协助象限 3 测试的各个方面(见图10-1),例如测试数据设置。本章的最后一节包含帮助团队专注于评估产品价值的重要方面的工具类型的示例。
It’s difficult to automate business-facing tests that critique the product, because such testing relies on human intellect, experience, and instinct. However, automated tools can assist with aspects of Quadrant 3 tests (see Figure 10-1), such as test data setup. The last section of this chapter contains examples of the types of tools that help teams focus on the important aspects of evaluating the product’s value.
虽然我们在本章中讨论的大部分测试都是手动的,但不要错误地认为这种手动测试足以生产出高质量的软件,并且您可以不自动化回归测试而侥幸逃脱。如果你没有在象限 1 和 2 中自动化测试,你将没有时间进行任何象限 3 测试。
While much of the testing we discuss in this chapter is manual, don’t make the mistake of thinking that this manual testing will be enough to produce high-quality software and that you can get away with not automating your regression tests. You won’t have time to do any Quadrant 3 tests if you haven’t automated the tests in Quadrants 1 and 2.
评估或批评产品是关于操纵被测系统并尝试重现最终用户的实际体验。了解不同的业务场景和工作流程有助于使体验更加真实。
Evaluating or critiquing the product is about manipulating the system under test and trying to recreate actual experiences of the end users. Understanding different business scenarios and workflows helps to make the experience more realistic.
我们建议尽早并经常向客户展示您正在开发的产品。一旦在故事开发过程中有基本的 UI 或报告可用,请将其展示给产品所有者或团队中的其他领域专家。然而,在迭代演示之前,并非业务方面的每个人都有机会看到迭代的可交付成果。迭代结束演示为业务用户和领域专家提供了一个机会,可以查看迭代中交付的内容并修改他们的优先级。这让他们有机会说:“这就是我所说的,但这不是我的意思。” 这是批评产品的一种形式。
We recommend showing customers what you’re developing early and often. As soon as a rudimentary UI or report is available during story development, show it to the product owner or other domain expert on the team. However, not everyone on the business side will get a chance to see the iteration’s deliverables until the iteration demo. End-of-iteration demonstrations are an opportunity for the business users and domain experts to see what has been delivered in the iteration and revise their priorities. It gives them a chance to say, “That’s what I said, but it’s not what I meant.” This is a form of critiquing the product.
向高管或高层管理人员进行演示也可以增强您对项目的信心。分阶段项目的缺点之一是直到最后才看到任何东西,管理层必须完全信任开发团队的报告。敏捷开发的增量和迭代特性使您有机会在生产时展示业务价值,甚至在发布之前。如果参与者积极询问有关新功能的问题,现场演示可能是一个非常强大的工具。
Demonstrations to the executives or upper management can instill confidence in your project as well. One of the downfalls of a phased project is there is nothing to see until the very end, and management has to place all of its trust in the development team’s reports. The incremental and iterative nature of agile development gives you a chance to demonstrate business value as you produce it, even before you release it. A live demonstration can be a very powerful tool if the participants are actively asking questions about the new features.
与其等到迭代结束,不如利用任何机会展示您的更改。Janet 最近参与的一个项目使用与业务用户的定期会议来演示新功能,以便获得即时反馈。任何需要的更改都被输入到下一次迭代中。
Rather than waiting until the end of the iteration, you can use any opportunity to demonstrate your changes. A recent project Janet worked on used regularly scheduled meetings with the business users to demonstrate new features in order to get immediate feedback. Any desired changes were fed into the next iteration.
选择适合您团队的演示频率,以便反馈循环足够快,以便您将更改合并到发布中。
Choose a frequency for your demonstrations that works for your team so that the feedback loop is quick enough for you to incorporate changes into the release.
Chapter 19, “Wrap Up the Iteration,” talks about end-of-iteration demonstrations and reviews.
非正式演示可能会更有成效。与业务专家坐下来,向她展示您的团队目前正在编写的故事。一起做一些探索性测试。我们听说过一些团队让他们的利益相关者在每次迭代演示后进行一些探索性测试,以帮助他们思考改进和未来的故事,以改变或构建刚刚交付的功能。
Informal demos can be even more productive. Sit down with a business expert and show her the story your team is currently coding. Do some exploratory testing together. We’ve heard of teams that get their stakeholders to do some exploratory testing after each iteration demo in order to help them think of refinements and future stories to change or build on the functionality just delivered.
业务用户可以帮助定义可以模仿最终用户行为的合理场景和工作流。现实生活中的领域知识对于创建准确的场景至关重要。我们想从头到尾测试系统,但不一定是黑盒子。
Business users can help define plausible scenarios and workflows that can mimic end user behavior. Real-life domain knowledge is critical to creating accurate scenarios. We want to test the system from end to end but not necessarily as a black box.
帮助团队了解业务和用户需求的一项好技术是“肥皂剧测试”,这是 Hans Buwalda [2004] 创造的一个术语。这里的想法是采用基于现实生活的场景,以类似于电视肥皂剧夸大行为和情绪的方式对其进行夸大,并将其压缩为一系列快速事件。想想这样的问题,“可能发生的最糟糕的事情是什么,它是如何发生的?”
One good technique for helping the team understand the business and user needs is “soap opera testing,” a term coined by Hans Buwalda [2004]. The idea here is to take a scenario that is based on real life, exaggerate it in a manner similar to the way TV soap operas exaggerate behavior and emotions, and compress it into a quick sequence of events. Think about questions like, “What’s the worst thing that can happen, and how did it happen?”
作为测试人员,我们经常“编造”测试数据,但它通常很简单,因此我们可以很容易地检查我们的结果。在测试不同的场景时,数据和流程都需要符合实际。查明数据是来自另一个系统还是手动输入的。如果可以的话,通过要求客户提供测试数据来获取样品。真实数据将流经系统,并可以沿途进行检查。在大型系统中,它的行为会根据所做的决定而有所不同。
As a tester, we often “make up” test data, but it is usually simple so we can easily check our results. When testing different scenarios, both the data and the flow need to be realistic. Find out if the data comes from another system or if it’s input manually. Get a sample if you can by asking the customers to provide data for testing. Real data will flow through the system and can be checked along the way. In large systems, it will behave differently depending on what decisions are made.
第 14 章,“敏捷测试自动化策略”,检查了获取测试数据的不同方法。
Chapter 14, “An Agile Test Automation Strategy,” examines different approaches to obtaining test data.
帮助定义场景和工作流程的工具可以很简单。数据流或过程流程图将有助于识别一些常见的场景。如果您花时间,这些场景可以帮助您思考复杂的问题。考虑用户及其动机。
Tools to help define the scenarios and workflows can be simple. Data flow or process flow diagrams will help identify some of the common scenarios. These scenarios can help you think through a complex problem if you take the time. Consider the users and their motivation.
端到端测试时,进行抽查以确保数据、状态标志、计算等按预期运行。使用流程图和其他视觉辅助工具来帮助您理解功能。许多组织依靠报告来做出决定,而这些报告似乎是我们核实的最后一件事。如果您的场景已被正确识别,您可能能够使用您的应用程序报告提供最终检查。
When testing end-to-end, make spot checks to make sure the data, status flags, calculations, and so on are behaving as expected. Use flow diagrams and other visual aids to help you understand the functionality. Many organizations depend on reports to make decisions, and those reports seem to be the last thing we verify. If your scenarios have been identified correctly, you might be able to use your application reports to provide a final check.
探索性测试 (ET) 是敏捷世界中测试的重要方法。作为调查工具,它是对故事测试和我们的自动回归套件的重要补充。这是一种无需脚本即可进行测试的复杂、深思熟虑的方法,它使您能够超越已经测试过的明显变化。探索性测试将学习、测试设计和测试执行结合到一种测试方法中。我们以一种有纪律的方式应用启发式方法和技术,以便“做”揭示出比仅仅思考问题更多的含义。在测试时,您可以了解有关被测系统的更多信息,并可以使用该信息来帮助设计新测试。
Exploratory testing (ET) is an important approach to testing in the agile world. As an investigative tool, it’s a critical supplement to the story tests and our automated regression suite. It is a sophisticated, thoughtful approach to testing without a script, and it enables you to go beyond the obvious variations that have already been tested. Exploratory testing combines learning, test design, and test execution into one test approach. We apply heuristics and techniques in a disciplined way so that the “doing” reveals more implications that just thinking about a problem. As you test, you learn more about the system under test and can use that information to help design new tests.
探索性测试不是通过详尽测试来评估软件的一种方法。它旨在为您的测试添加另一个维度。您所做的只是足以查看“已完成”的故事是否真的让您满意。
Exploratory testing is not a means of evaluating the software through exhaustive testing. It is meant to add another dimension to your testing. You do just enough to see if the “done” stories are really done to your satisfaction.
参考书目列出了您应该调查的更多资源,以了解有关探索性测试的更多信息。
The bibliography lists more resources you should investigate to learn more about exploratory testing.
探索性测试的一个有价值的副作用是由此产生的学习。它揭示了可以使用更多自动化测试的产品领域,并提出了新功能或修改功能的想法,从而产生了新的故事。
A valuable side effect of exploratory testing is the learning that comes out of it. It reveals areas of the product that could use more automated tests and brings up ideas for new or modified features that lead to new stories.
不熟悉探索性测试的人经常将其与临时测试混淆。探索性测试不是坐在键盘前打字。不熟练的“黑盒”测试人员可能不知道如何进行探索性测试。
People unfamiliar with exploratory testing often confuse it with ad hoc testing. Exploratory testing isn’t sitting down at a keyboard and typing away. Unskilled “black box” testers may not know how to do exploratory testing.
探索性测试从一个章程开始,该章程说明将探索功能的哪些方面。它需要批判性思维、解释结果并将其与预期或类似系统进行比较。测试时跟随“气味”是一个重要组成部分。测试人员在探索性测试期间做笔记,以便他们可以重现他们看到的任何问题并根据需要进行更多调查。
Exploratory testing starts with a charter of what aspects of the functionality will be explored. It requires critical thinking, interpreting the results, and comparing them to expectations or similar systems. Following “smells” when testing is an important component. Testers take notes during their exploratory testing sessions so that they can reproduce any issues they see and do more investigation as needed.
探索性测试一词因测试的“上下文驱动学派”而流行。这是一项纪律严明的活动,可以学习。基于会话的测试管理是一种测试方法,旨在使探索性测试可审计和可测量 [ Bach,2003 年]。
The term exploratory testing was popularized by the “context-driven school” of testing. It’s a highly disciplined activity, and it can be learned. Session-based test management is one method of testing that’s designed to make exploratory testing auditable and measurable [Bach, 2003].
基于会话的测试结合了问责制和探索性测试。它为测试人员的探索性测试体验提供了一个框架,以便他们能够以一致的方式报告结果。
Session-based testing combines accountability and exploratory testing. It gives a framework to a tester’s exploratory testing experience so that they can report results in a consistent way.
就像先把外面的部分拼在一起来解决拼图游戏一样,我们可以使用基于会话的测试来为我们提供工作框架。在基于会话的测试中,我们创建一个任务或章程,然后为我们的会话设定时间限制,这样我们就可以专注于重要的事情。作为测试人员,我们经常会偏离轨道并最终追逐一个对我们当前正在测试的内容可能重要或不重要的错误。
Like solving the jigsaw puzzle by putting together the outside pieces first, we can use session-based testing to give us the framework in which we work. In session-based testing, we create a mission or a charter and then time-box our session so we can focus on what’s important. Too often as testers, we can go off track and end up chasing a bug that might or might not be important to what we are currently testing.
会话分为三种任务:测试设计和执行、错误调查和报告以及会话设置。我们衡量在设置上花费的时间与实际测试执行上花费的时间,以便我们知道在哪些方面花费的时间最多。我们可以以一致的方式捕获结果,以便我们可以向团队报告。
Sessions are divided into three kinds of tasks: test design and execution, bug investigation and reporting, and session setup. We measure the time we spend on setup versus actual test execution so that we know where we spend the most time. We can capture results in a consistent manner so that we can report back to the team.
我们也可以将探索性测试与测试自动化结合起来。Jonathan Kohl 在他的文章“Man and Machine”[2007] 中谈到了交互式测试自动化以协助探索性测试。使用自动化来进行测试设置、数据生成、重复性任务,或者沿着工作流程前进到您想要开始的地方。然后你开始使用你的测试技能和经验来找到真正“好”的错误,那些以其他方式逃避注意的阴险的错误。您还可以使用自动化测试套件进行探索。稍微修改一下,观察运行结果,再次修改,观察会发生什么。
We can combine exploratory testing with test automation as well. Jonathan Kohl, in his article “Man and Machine” [2007], talks about interactive test automation to assist exploratory testing. Use automation to do test set up, data generation, repetitive tasks, or to progress along a workflow to the place you want to start. Then you start using your testing skills and experience to find the really “good” bugs, the insidious ones that otherwise escape attention. You can also use an automated test suite to explore. Just modify it a bit, watch the results as it runs, modify it again, and watch what happens.
通过探索性测试,每个测试人员对问题都有不同的方法,并且具有独特的工作风格。但是,有某些属性可以成为一个好的探索性测试人员。一个好的测试员:
With exploratory testing, each tester has a different approach to a problem, and has a unique style of working. However, there are certain attributes that make for a good exploratory tester. A good tester:
是系统的,但追求“气味”(异常,不一致的片段)
Is systematic, but pursues “smells” (anomalies, pieces that aren’t consistent)
通过使用 Oracles 学习识别问题(我们识别问题的原理或机制)
Learns to recognize problems through the use of Oracles (principle or mechanism by which we recognize a problem)
Chooses a theme or role or mission statement to focus testing
时间限制会议和边路旅行
Time-boxes sessions and side trips
考虑专家或新手用户会做什么
Thinks about what the expert or novice user would do
与领域专家一起探索
Explores together with domain experts
检查类似或有竞争力的应用程序
Checks out similar or competitive applications
探索性测试帮助我们了解应用程序的行为。测试人员通常对他们正在测试的应用程序了解很多。他们如何判断该应用程序是否可供技术水平较低或不熟悉的用户使用?可用性测试对于许多软件系统来说都是至关重要的。我们将在下一节中讨论。
Exploratory testing helps us learn about the behavior of an application. Testers generally know a lot about the application they’re testing. How do they judge whether the application is usable by users who are less technical or not familiar with it? Usability testing is vital for many software systems. We’ll talk about that in the next section.
有两种类型的可用性测试。第一种类型是由用户体验人员预先完成的类型,使用线框等工具来帮助驱动编程。这些类型的测试属于第 2 象限。在本节中,我们将讨论批评产品的可用性测试类型。我们使用角色等工具和我们的直觉来帮助我们在考虑最终用户的情况下看待产品。
There are two types of usability testing. The first type is the kind that is done up front by the user experience folks, using tools such as wire frames to help drive programming. Those types of tests belong in Quadrant 2. In this section, we’re talking about the kind of usability testing that critiques the product. We use tools such as personas and our intuition to help us look at the product with the end user in mind.
让我们看一个网上购物的例子。我们考虑谁将使用该网站。是以前网上购物过的人,还是不知道如何进行的全新用户?我们猜测它将是两者的混合体,以及其他的。花时间询问您的营销团队以获取最终用户的人口统计数据。这些数字可能会帮助您计划测试。
Let’s look at an online shopping example. We think about who will use the site. Will it be people who have shopped online before, or will it be brand new users who have no idea how to proceed? We’re guessing it will be a mixture of both, as well as others. Take the time to ask your marketing group to get the demographics of the end users. The numbers might help you plan your testing.
使用角色的一种方法是让您的团队为您的应用程序发明几个不同的用户,代表不同的经验水平和需求。对于我们的 Internet 零售应用程序,我们可能具有以下角色:
One approach to using personas is for your team to invent several different users of your application representing different experience levels and needs. For our Internet retail application, we might have the following personas:
Nancy Newbie,一位刚接触网络购物并担心身份被盗用的老年人
Nancy Newbie, a senior citizen who is new to Internet shopping and nervous about identity theft
Hudson Hacker,寻找欺骗结帐页面的方法
Hudson Hacker, who looks for ways to cheat the checkout page
Enrico Executive,他在网上购物并向全球所有客户寄送礼物
Enrico Executive, who does all his shopping online and ships gifts to all his clients worldwide
Betty Bargain,正在寻找超值优惠
Betty Bargain, who’s looking for great deals
黛比·迪瑟勒 (Debbie Ditherer),她很难决定自己真正想要订购哪些商品
Debbie Ditherer, who has a hard time deciding what items she really wants to order
我们可能会在我们的工作区域挂上代表这些不同角色的照片和他们的传记,以便我们始终牢记它们。我们可以依次测试每个角色相同的场景,看看他们可能会遇到什么不同的体验。
We might hang photos representing these different personas and their biographies in our work area so that we always keep them in mind. We can test the same scenario as each persona in turn and see what different experiences they might encounter.
我们从 Brian Marick 和 Elisabeth Hendrickson 那里学到的另一种进行角色测试的方法是选择一个虚构人物或名人,并想象他们将如何使用我们的应用程序。英格兰女王能否驾驭我们的结帐流程?Homer Simpson 将如何搜索他想要的物品?
Another way to approach persona testing, which we learned from Brian Marick and Elisabeth Hendrickson, is to pick a fictional character or famous celebrity and imagine how they would use our application. Would the Queen of England be able to navigate our checkout process? How might Homer Simpson search for the item he wants?
在探索应用程序时,您也可以只担任新手、中级和专家用户的角色。用户能否在没有说明的情况下弄清楚他们应该做什么?如果您有很多新用户,您可能需要使界面非常简单。
You can also just assume the roles of novice, intermediate, and expert users as you explore the application. Can users figure out what they are supposed to do without instructions? If you have a lot of first-time users, you might need to make the interface very simple.
如果您的应用程序是为特定类型的用户定制的,它可能需要“智能”而不是直观。培训课程可能足以让克服最初缺乏可用性的问题,以便可以将界面设计为最大的效率和实用性。
If your application is custom-built for specific types of users, it might need to be “smart” rather than intuitive. Training sessions might be sufficient to get over the initial lack of usability so that the interface can be designed for maximum efficiency and utility.
导航是可用性测试的另一个方面。测试链接并确保 Tab 键顺序合理非常重要。如果用户可以选择应用程序或网站,并且初次体验不佳,他们可能不会再次使用您的应用程序。其中一些测试是可自动化的,但测试实际用户体验很重要。
Navigation is another aspect of usability testing. It’s incredibly important to test links and make sure the tabbing order makes sense. If a user has a choice of applications or websites, and has a bad first experience, they likely won’t use your application again. Some of this testing is automatable, but it’s important to test the actual user experience.
如果您有权访问最终用户,请让他们参与测试导航。与真实用户配对,或观看用户实际使用该应用程序并做笔记。在设计新的用户界面时,请考虑使用焦点小组来评估不同的界面。您可以从在纸上绘制的模型和流程开始,征求意见,然后尝试 HTML 模型,以获得早期反馈。
If you have access to the end users, get them involved in testing the navigation. Pair with a real user, or watch one actually use the application and take notes. When you’re designing a new user interface, consider using focus groups to evaluate different interfaces. You can start with mock-ups and flows drawn on paper, get opinions, and try HTML mock-ups next, to get early feedback.
请参阅第 8 章“支持团队的面向业务的测试”,以了解绿野仙踪测试的示例,这是一种可用性设计方法。
See Chapter 8, “Business-Facing Tests that Support the Team,” for an example of Wizard of Oz testing, which is one approach to designing for usability.
在评估您的应用程序的可用性时,请考虑其他类似的应用程序。他们如何完成任务?您认为它们是用户友好的还是直观的?如果您可以访问竞争软件,请花一些时间研究这些应用程序的工作原理并将它们与您的产品进行比较。例如,您正在测试一个采用日期范围的用户界面,并且它具有用于选择日期的弹出式日历功能。看看类似的日历功能如何在航空公司预订网站上工作。
When evaluating your application for usability, think about other applications that are similar. How do they accomplish tasks? Do you consider them user-friendly or intuitive? If you can get access to competing software, take some time to research how those applications work and compare them with your product. For example, you’re testing a user interface that takes a date range, and it has a pop-up calendar feature to select the date. Take a look at how a similar calendar function works on an airline reservation website.
可用性测试是一个相当专业的领域。如果您正在制作一个内部应用程序供一些接受过使用培训的用户使用,您可能不需要在可用性测试上投入太多。如果您正在为一家电话公司编写在线目录帮助,可用性可能是您的主要关注点,因此您需要尽可能多地了解它,或者请一位可用性专家。
Usability testing is a fairly specialized field. If you’re producing an internal application to be used by a few users who will be trained in its use, you probably don’t need to invest much in usability testing. If you’re writing the online directory assistance for a phone company, usability might be your main focus, so you need to learn as much as you can about it, or bring in a usability expert.
在题为“人与机器”[2007 年] 的演讲中,Jonathan Kohl 谈到了测试界面的替代方案。与其总是考虑通过用户界面进行测试,不如考虑以其他方式解决问题。考虑从您可以接近的各个角度测试整个系统。考虑使用模拟器或仿真器等工具。
In a presentation titled “Man and Machine” [2007], Jonathan Kohl talked about alternatives for testing interfaces. Instead of always thinking about testing through the user interface, consider attacking the problem in other ways. Think about testing the whole system from every angle that you can approach. Consider using tools like simulators or emulators.
第 9 章“支持团队的面向业务的测试工具包”提供了有关促进这些测试的工具的更多详细信息。
Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” provides more detail about tools that facilitate these tests.
在第 8 章和第9 章中,我们讨论了 GUI 背后的测试以驱动开发。在本节中,我们将展示您可以扩展 API 测试以尝试不同的排列和组合。
In Chapter 8 and Chapter 9, we talked about testing behind the GUI to drive development. In this section, we show that you can extend your tests for the API in order to try different permutations and combinations.
API (应用程序编程接口)是可以由其他软件应用程序或组件执行的功能的集合。最终用户通常永远不会意识到 API 的存在;她只是与顶部的界面进行交互。
An API (application programming interface) is a collection of functions that can be executed by other software applications or components. The end user is usually never aware that an API exists; she simply interacts with the interface on top.
每个 API 调用都有一个特定的函数,其中包含许多接受不同输入的参数。每个变体都会返回不同的结果。简单的测试是简单的输入。当参数一起工作以提供许多可能的变化时,会出现更复杂的测试模式。有时参数是可选的,因此了解可能性很重要。对于输入和预期结果,还应考虑边界条件。例如,对参数使用有效和无效字符串,改变内容,改变字符串输入的长度。
Each API call has a specific function with a number of parameters that accept different inputs. Each variation will return a different result. The easy tests are simple inputs. The more complicated testing patterns occur when the parameters work together to give many possible variations. Sometimes parameters are optional, so it’s important that you understand the possibilities. Boundary conditions should be considered as well, for both the inputs and expected results. For example, use both valid and invalid strings for parameters, vary the content, and vary the length of the strings’ input.
另一种测试方法是改变 API 调用的顺序。更改顺序可能会产生意想不到的结果,并揭示通过 UI 测试永远不会发现的错误。与使用 UI 相比,您可以更轻松地控制测试。
Another way to test is to vary the order of the API calls. Changing the sequence might produce unexpected results and reveal bugs that would never be found through UI testing. You can control the tests much more easily than when using the UI.
API 调用可以在应用程序生命周期的早期开发,这意味着测试也可以在早期进行。通过 API 进行测试可以在开发 UI 之前增强对系统的信心。因为这种类型的测试可以自动化,所以您需要与您的程序员一起了解所有参数和每个函数的用途。如果您的程序员或自动化团队开发了易于使用的测试工具,您应该能够有条不紊地创建一套测试用例来执行该功能。
API calls can be developed early in an application life cycle, which means testing can occur early as well. Testing through an API can give confidence in the system before a UI is ever developed. Because this type of testing can be automated, you will need to work with your programmers to understand all of the parameters and the purpose of each function. If your programmers or automation team develop a test harness that is easy to use, you should be able to methodically create a suite of test cases that exercises the functionality.
Web 服务是一种基于服务的体系结构,它提供一个外部接口以便其他人可以访问系统。可能有多个利益相关者,您甚至可能不知道谁会使用您的产品。您的测试需要确认外部客户期望的服务质量。
Web services are a services-based architecture that provides an external interface so that others can access the system. There might be multiple stakeholders, and you may not even know who will be using your product. Your testing will need to confirm the quality of service that the external customers expect.
Web 服务通常需要大量的安全、压力和可靠性测试。有关这些类型的测试的更多信息,请参阅第 12 章“使用面向技术的测试来评价产品”。
Web services generally require plenty of security, stress, and reliability testing. See Chapter 12, “Critiquing the Product using Technology-Facing Tests,” for more on these types of tests.
在创建测试计划时考虑向客户承诺的服务级别。花时间进行探索性测试,以模拟用户可能访问 Web 服务的不同方式。
Consider levels of service that have been promised to clients when you are creating your test plans. Make time for exploratory testing to simulate the different ways users might access the web services.
Web 服务标准的使用还为当前的测试工具提供了其他含义。与 API 调用一样,基于 Web 服务的集成突出了验证接口点的重要性。但是,我们还需要考虑消息格式和处理、排队时间和消息响应时间。
The use of web services standards also offers other implications for current testing tools. As with API calls, web services-based integration highlights the importance of validating interface points. However, we also need to consider message formats and processing, queuing times, and message response times.
使用利用 GUI 驱动的自动化的测试工具对于 Web 服务项目来说是完全不够的。一种在“幕后”封装实现细节的特定领域语言非常适合测试 Web 服务。
Using testing tools that utilize GUI-driven automation is simply inadequate for a web services project. A domain-specific language that encapsulates implementation details “behind the scenes” works well for testing web services.
第 12 章,“测试象限总结”,有一个测试 Web 服务的例子。
Chapter 12, “Summary of Testing Quadrants,” has an example of testing web services.
在测试过程中经常被忽视的系统组件之一是文档。作为敏捷开发人员,我们可能更看重可工作的软件而不是文档,但我们仍然看重文档!用户手册和在线帮助与软件一样需要验证。您的团队可能会聘请专家,例如创建和验证文档的技术作家。与产品的所有其他组件一样,您的整个团队对文档的质量负责,包括硬拷贝和电子文档。
One of the components of the system that is often overlooked during testing is documentation. As agile developers, we may value working software over documentation, but we still value documentation! User manuals and online help need validation just as much as software. Your team may employ specialists such as technical writers who create and verify documentation. As with all other components of the product, your whole team is responsible for the quality of the documentation, and that includes both hard copy and electronic.
您的团队可能会进行象限 2 测试以支持团队制作文档;事实上,我们鼓励这样做。Lisa 的团队编写的代码生成的文档的内容由政府法规指定,程序员可以先测试编写大部分代码。但是,自动化测试很难判断文档的格式是否正确或使用了可读的字体。他们也无法评估用户手册等文档的内容是否准确或有用。因为文档有很多主观成分,所以验证它更像是一种批评活动。
Your team might do Quadrant 2 tests to support the team as they produce documentation; in fact we encourage it. Lisa’s team writes code that produces documents whose contents are specified by government regulations, and programmers can write much of the code test-first. However, it’s difficult for automated tests to judge whether a document is formatted correctly or uses a readable font. They also can’t evaluate whether the contents of documents such as user manuals are accurate or useful. Because documentation has many subjective components, validating it is more of a critiquing activity.
也不要忘记查看帮助文本。帮助文本的链接是否易于识别?它们在整个用户界面中是否一致?帮助文本是否清晰呈现?如果它在弹出窗口中打开,并且用户在他们的浏览器中阻止弹出窗口,会有什么影响?帮助是否涵盖了所有需要的主题?在 Lisa 的项目中,帮助文本的优先级往往较低,因此通常根本无法完成。这是一项业务决策,但如果您觉得应用程序的某个区域需要额外的帮助文本或文档,请向您的团队和客户提出问题。
Don’t forget to check the help text too. Are the links to help text easily identifiable? Are they consistent throughout the user interface? Is the help text presented clearly? If it opens in a pop-up, and users block pop-ups in their browsers, what’s the impact? Does the help cover all of the topics needed? On Lisa’s projects, help text tends to be a low priority, so it often doesn’t get done at all. That’s a business decision, but if you feel an area of the application needs extra help text or documentation, raise the issue to your team and your customers.
从测试的角度来看,另一个经常被忽视的系统组件是报告。出于决策目的,报告对许多用户来说至关重要,但通常留到最后,要么没有完成,要么执行不力。可以定制报告以满足特定的客户需求,但是有许多第三方工具可用于生成报告。报告可以是应用程序本身的一部分,也可以通过单独的报告系统为最终用户生成。
Another system component that’s often overlooked from a testing perspective is reports. Reports are critical to many users for decision-making purposes but are often left until the very end, and either don’t get done or are poorly executed. Reports might be tailored to meet specific customer needs, but there are many third-party tools available for generating reports. Reports may be part of the application itself or be generated through a separate reporting system for end users.
我们与其他象限 3 测试活动一起讨论测试报告,以评价产品,但我们建议您也编写象限 2 报告测试,以指导编码并帮助团队在生成报告时了解客户的需求。它们当然可以写成测试先行的。但是,与文档一样,您需要查看报告以了解它是否足够容易阅读并以易于理解的方式呈现信息。
We discuss testing reports along with the other Quadrant 3 test activities in order to critique the product, but we recommend that you also write Quadrant 2 report tests that will guide the coding and help the team understand the customer’s needs as it produces reports. They can certainly be written test-first. Like documents, though, you need to look at a report to know if it’s easy enough to read and presents information in an understandable way.
测试报告时最大的挑战之一不是格式化,而是获取正确的数据。当您尝试为报告创建测试数据时,可能很难获得真实数据的良好横截面。导致报告失败的通常也是边缘情况,因此合并这些额外数据是不可行的。在大多数情况下,最好使用生产数据(或从生产系统复制到测试环境的数据)来测试不同的报告变体。
One of the biggest challenges when testing reports is not the formatting but getting the right data. When you try to create test data for reports, it can be difficult to get a good cross section of realistic data. It also is usually the edge cases that make the reports fail, so incorporating that extra data is not feasible. In most cases, it’s best to use production data (or data copied from the production system into a test environment) to test the different reporting variations.
如果您正在处理一个涉及大量报告的项目,请不要屈服于将它们留到最后的诱惑。如果可以的话,在每次迭代中包含一些报告。一份报告可以是一个故事,也可以分解成几个故事。使用模型帮助客户决定报告内容和格式。在报告中找到“薄切片”或“关键路径”,首先对其进行编码,并在添加下一个切片之前将其展示给您的客户。增量开发与报表一样适用于其他软件。
If you’re tackling a project that involves lots of reports, don’t give in to the temptation to leave them to the end. Include some reports in each iteration if you can. One report could be a single story or maybe even broken up into a couple of stories. Use mock-ups to help the customers decide on report contents and formatting. Find the “thin slice” or “critical path” in the report, code that first, and show it to your customer before you add the next slice. Incremental development works as well with reports as it does with other software.
有关使用薄切片的信息,请参阅第 8 章“支持团队的面向业务的测试”。
See Chapter 8, “Business-Facing Tests that Support the Team,” for information about using thin slices.
有时,您的客户自己并不确定报告的外观或如何逐步处理报告。有时团队中没有人预料到测试工作会有多困难。
Sometimes your customers themselves aren’t sure how a report should look or how to approach it incrementally. And sometimes nobody on the team anticipates how hard the testing effort will prove to be.
短迭代意味着很难腾出时间进行充分的探索性测试和其他象限 3 活动。让我们看看可能有助于加快此测试并为重要的手动和可视化测试腾出时间的工具。
Short iterations mean that it can be hard to make time for adequate exploratory testing and other Quadrant 3 activities. Let’s look at tools that might help speed up this testing and make time for vital manual and visual tests.
探索性测试是手动测试。一些最好的测试之所以发生,是因为一个人正在关注细节,而如果我们关注这些细节,这些细节往往会被遗漏一个脚本。直觉是我们无法让机器学习的东西。然而,有许多工具可以帮助我们追求卓越。
Exploratory testing is manual testing. Some of the best testing happens because a person is paying attention to details that often get missed if we are following a script. Intuition is something that we cannot make a machine learn. However, there are many tools that can assist us in our quest for excellence.
工具不应该取代人际互动;他们应该增强体验。工具可以为测试人员提供更多的能力来发现难以重现的错误,这些错误通常会被归档,因为没有人能够处理它们。探索性测试是非常规的,那么为什么工具不能如此呢?考虑可以将工具整合到您的测试中的省力、高价值的方式。
Tools shouldn’t replace human interaction; they should enhance the experience. Tools can provide testers with more power to find the hard-to-reproduce bugs that often get filed away because no one can get a handle on them. Exploratory testing is unconventional, so why shouldn’t the tools be as well? Think about low-effort, high-value ways that tools can be incorporated into your testing.
计算机擅长执行重复性任务和执行计算。这是它们比人类强得多的两个领域,所以让我们用它们来完成这些任务。因为测试需要跟上编码的步伐,所以我们可以获得的任何时间优势都是一种奖励。
Computers are good at doing repetitive tasks and performing calculations. These are two areas where they are much better than humans, so let’s use them for those tasks. Because testing needs to keep pace with coding, any time advantage we can gain is a bonus.
请参阅参考书目,了解 Jonathan Kohl 关于结合使用人力和自动化力量进行最佳测试的著作。
See the bibliography for references to Jonathan Kohl’s writings on using human and automation power together for optimal testing.
在接下来的几节中,我们将研究自动化可以利用探索性测试的一些领域。我们涵盖的内容是测试设置、测试数据生成、监控、模拟器和模拟器。
In the next few sections, we’ll look at some areas where automation can leverage exploratory testing. The ones we cover are test setup, test data generation, monitoring, simulators, and emulators.
让我们想想我们在测试时做了什么。我们刚刚发现了一个错误,但不是一个容易重现的错误。我们非常确定它是组件之间交互的结果。我们回到起点,尝试一个又一个场景。很快我们就花了一整天的时间来重现这个错误。
Let’s think about what we do when we test. We’ve just found a bug, but not one that is easily reproducible. We’re pretty sure it happens as a result of interactions between components. We go back to the beginning and try one scenario after another. Soon we’ve spent the whole day just trying to reproduce this one bug.
问问自己如何使这更容易。我们发现最耗时的任务之一是测试设置和为实际测试找到正确的起点。如果您使用基于会话的测试,那么您已经知道您花了多少时间来设置测试,因为您一直在跟踪那个特定的时间浪费。这是一些自动化的绝佳机会。
Ask yourself how you can make this easier. We’ve found that one of the most time-consuming tasks is the test setup and getting to the right starting point for your actual test. If you use session-based testing, then you already know how much time you spend setting up the test, because you have been tracking that particular time waster. This is an excellent opportunity for some automation.
用于支持第 9 章中描述的团队的面向业务的测试的工具对于手动探索性测试也很有价值。可以运行自动化功能测试脚本来设置数据和场景以启动探索性测试会话。配置为接受运行时参数的测试对于设置评估产品的起点特别有用。
The tools used for business-facing tests that support the team described in Chapter 9 are also valuable for manual exploratory testing. Automated functional test scripts can be run to set up data and scenarios to launch exploratory testing sessions. Tests configured to accept runtime parameters are particularly powerful for setting up a starting point for evaluating the product.
您用于功能回归测试和指导开发的测试脚本并不是唯一可以帮助您摆脱单调乏味的手动探索性测试的工具。还有其他工具可帮助设置测试数据以及帮助您评估测试会话的输出。
The test scripts you use for functional regression testing and for guiding development aren’t the only tools that help take the tedium out of manual exploratory testing. There are other tools to help set up test data as well as to help you evaluate the outputs of your testing sessions.
无论您使用的是什么工具,请考虑如何调整它以在插入不同输入的情况下一遍又一遍地运行场景。Janet 还成功地使用 Ruby 和 Watir 来设置测试以运行多次以帮助识别错误。以与最终用户大致相同的方式驱动浏览器或 UI 的工具将使您的测试更加可靠,因为您可以在显示器上回放它并观察在设置过程中可能看起来不正常的任何东西。当你到达测试真正开始的地方时,你就可以利用你出色的测试能力来追踪错误的来源。
Whatever tool you are using, think about how it can be adapted to run the scenario over and over with different inputs plugged in. Janet has also successfully used Ruby with Watir to set up tests to run multiple times to help identify bugs. Tools that drive the browser or UI in much the same way that an end user would makes your testing more reliable because you can play it back on your monitor and watch for anything that might not look as it should during the setup. When you get to the place where the test actually starts, you can then use your excellent testing abilities to track down the source of the bug.
PerlClip 是一个工具示例,您可以使用它来测试具有不同类型输入的文本字段。James Bach 在他的网站www.satisfice.com上免费提供它,它对验证字段非常有帮助。例如,如果您有一个最多接受 200 个字符输入的字段,手动测试该字段及其边界将非常繁琐。使用 PerlClip 创建一个字符串,将其放入您的自动化库中,然后让您的自动化工具调用该字符串来测试该值。
PerlClip is an example of a tool that you can use to test a text field with different kinds of inputs. James Bach provides it free of charge on his website, www.satisfice.com, and it can be very helpful in validating fields. For example, if you have a field that will accept a maximum input of 200 characters, testing this field and its boundaries manually would be very tedious. Use PerlClip to create a string, put it in your automation library, and have your automation tool call the string to test the value.
Unix/Linux 命令tail -f或 James Bach 的 LogWatch 等工具可以帮助监视日志文件中的错误情况。IDE 还提供日志分析工具。许多错误消息永远不会显示在屏幕上,因此如果您通过 GUI 进行测试,您永远不会看到它们。熟悉这些工具,因为它们可以使您的测试更加有效和高效。如果你不是要确定您的系统在哪里记录警告和错误,请询问您的开发人员。他们可能对如何监控系统有很多想法。
Tools like the Unix/Linux command tail -f, or James Bach’s LogWatch, can help monitor log files for error conditions. IDEs also provide log analysis tools. Many error messages are never displayed on the screen, so if you’re testing via the GUI, you never see them. Get familiar with tools like these, because they can make your testing more effective and efficient. If you are not sure where your system logs warnings and errors, ask your developers. They probably have lots of ideas about how you can monitor the system.
模拟器是用于创建数据的工具,这些数据代表被测系统的真实数据的关键特征和行为。如果您无法访问系统的真实数据,模拟数据有时也几乎可以正常工作。使用模拟器的另一个优势是随着时间的推移将数据泵入系统。它可用于帮助生成在正常情况下难以创建的错误条件,并可减少边界测试的时间。
Simulators are tools used to create data that represent key characteristics and behavior of real data for the system under test. If you do not have access to real data for your system, simulated data will sometimes work almost as well. The other advantage of using a simulator is for pumping data into a system over time. It can be used to help generate error conditions that are difficult to create under normal circumstances and can reduce time in boundary testing.
请参阅第 12 章“测试象限总结”中的“系统测试”示例,了解模拟器对于测试整个系统的重要性。
See the “System Test” example in Chapter 12, “Summary of Testing Quadrants,” to see how a simulator was critical to the testing the whole system.
设置数据和测试场景只是成功的一半。您还需要有一种方法来观察您的测试结果。让我们考虑一些用于此目的的工具。
Setting up data and test scenarios is half of the picture. You also need to have a way to watch the outcomes of your testing. Let’s consider some tools for that purpose.
仿真器复制系统的功能,使其表现得像被测系统。使用模拟器的原因有很多。当您需要测试与其他系统或设备接口的代码时,仿真器是非常宝贵的。
An emulator duplicates the functionality of a system so that it behaves like the system under test. There are many reasons to use an emulator. When you need to test code that interfaces with other systems or devices, emulators are invaluable.
仿真器是一种有助于保持测试和编码同步进行的工具。使用它们是测试以跟上短迭代开发的一种方法。在计划发布和迭代时,请考虑可能有助于创建类似生产的测试场景的工具类型。看看您是否可以使用您已经用于自动化测试的工具来推动开发,作为探索性测试的辅助工具。
Emulators are one tool that helps to keep testing and coding moving together hand-in-hand. Using them is one way for testing to keep up with development in short iterations. As you plan your releases and iterations, think about the types of tools that might help with creating production-like test scenarios. See if you can use the tools you’re already using for automating tests to drive development as aids to exploratory testing.
通过测试推动开发对于任何项目的成功都是至关重要的。然而,我们人类不会总是完全正确地获得所需系统行为的所有要求。当我们的业务专家提供功能应该如何工作的示例时,他们自己可能会错过功能的重要方面或与系统其他部分的交互。我们必须使用技术来帮助客户和开发团队更多地了解系统,以便他们能够不断改进产品。
Driving development with tests is critical to any project’s success. However, we humans won’t always get all of the requirements for desired system behavior entirely correct. Our business experts themselves can miss important aspects of functionality or interaction with other parts of the system when they provide examples of how a feature should work. We have to use techniques to help both the customer and developer teams learn more about the system so they can keep improving the product.
测试工作的很大一部分用于从业务角度对产品进行评价。本章为您提供了一些关于可以执行的测试类型的想法,以使您的测试工作更有效。
A large part of the testing effort is spent critiquing the product from a business perspective. This chapter gave you some ideas about the types of tests you can do to make your testing efforts more effective.
向利益相关者展示软件以获得早期反馈,这将有助于指导构建正确的东西。
Demonstrate software to stakeholders in order to get early feedback that will help direct building the right stuff.
使用场景和工作流从头到尾测试整个系统。
Use scenarios and workflows to test the whole system from end to end.
使用探索性测试来补充自动化并利用人类的智慧和感知。
Use exploratory testing to supplement automation and to take advantage of human intellect and perceptions.
如果在测试和编码时不考虑可用性,应用程序可能会变成货架软件。始终注意系统的使用方式。
Without usability in mind when testing and coding, applications can become shelfware. Always be aware of how the system is being used.
在 GUI 后面进行测试是获得应用程序功能的最有效方法。做一些研究以了解如何处理您的应用程序。
Testing behind the GUI is the most effective way of getting at the application functionality. Do some research to see how you can approach your application.
结合各种测试来制作一个好的回归套件。
Incorporate all kinds of tests to make a good regression suite.
不要忘记测试文档和报告。
Don’t forget about testing documentation and reports.
自动化工具可以执行繁琐和重复的任务,例如数据和测试场景设置,并腾出更多时间用于重要的手动探索性测试。
Automation tools can perform tedious and repetitive tasks, such as data and test scenario setup, and free up more time for important manual exploratory testing.
您已经用于自动化功能测试的工具也可能有助于利用探索性测试。
Tools you’re already using to automate functional tests might also be useful to leverage exploratory tests.
操作系统和 IDE 中内置的监视、资源使用和日志分析工具可帮助测试人员评估应用程序的行为。
Monitoring, resource usage, and log analysis tools built into operating systems and IDEs help testers appraise the application’s behavior.
即使您无法复制确切的生产环境,模拟器和仿真器也可以进行探索性测试。
Simulators and emulators enable exploratory testing even when you can’t duplicate the exact production environment.
即使使用测试来推动开发,也可能会遗漏或误解对所需行为或与其他系统交互的要求。象限 3 的活动帮助团队不断为产品增加价值。
Even when tests are used to drive development, requirements for desired behavior or interaction with other systems can be missed or misunderstood. Quadrant 3 activities help teams keep adding value to the product.
本章的重点是我们测试象限的右下角。我们研究了通过面向业务和面向技术的测试来推动开发。编写代码后,我们不再推动开发,而是寻找评价产品的方法。在上一章中,我们研究了从商业角度进行批评的方法。现在我们看看如何从面向技术的角度进行批评。这些测试是评估我们的产品是否提供正确商业价值的重要手段。
This chapter is focused on the bottom right corner of our testing quadrant. We’ve looked at driving development with both business-facing and technology-facing tests. After the code is written, we are no longer driving the development but are looking at ways to critique the product. In the previous chapter, we examined ways to critique from a business point of view. Now we look at ways to critique from a technology-facing point of view. These tests are an important means of evaluating whether our product delivers the right business value.
单个故事是拼图的一部分,但应用程序的意义远不止于此。批评产品的面向技术的测试更多与功能需求相比,更关注非功能需求。我们从技术角度担心产品存在缺陷。我们没有使用业务领域语言,而是使用编程领域词汇表来描述需求。这是象限 4 的省份(见图11-1)。
Individual stories are pieces of the puzzle, but there’s more to an application than that. The technology-facing tests that critique the product are more concerned with the nonfunctional requirements than the functional ones. We worry about deficiencies in the product from a technical point of view. Rather than using the business domain language, we describe requirements using a programming domain vocabulary. This is the province of Quadrant 4 (see Figure 11-1).
非功能性需求包括配置问题、安全性、性能、内存管理、“能力”(例如,可靠性、互操作性和可扩展性)、恢复,甚至数据转换。并非所有项目都关心所有这些问题,但最好有一个清单以确保团队考虑这些问题并询问客户每个问题的重要性。
Nonfunctional requirements include configuration issues, security, performance, memory management, the “ilities” (e.g., reliability, interoperability, and scalability), recovery, and even data conversion. Not all projects are concerned about all of these issues, but it is a good idea to have a checklist to make sure the team thinks about them and asks the customer how important each one is.
我们的客户应该考虑所有重要的质量属性和因素,并做出明智的权衡。然而,许多客户关注在应用程序的业务方面,不理解许多非功能性需求在帮助定义产品所需质量水平方面的重要性。他们可能会假设开发团队只会处理性能、可靠性和安全性等问题。
Our customer should think about all of the quality attributes and factors that are important and make informed trade-offs. However, many customers focus on the business side of the application and don’t understand the criticality of many nonfunctional requirements in their role of helping to define the level of quality needed for the product. They might assume that the development team will just take care of issues such as performance, reliability, and security.
我们认为开发团队有责任解释不解决这些非功能性或跨功能需求的后果。我们实际上都是想要提供良好价值的产品团队的一员,而这些以技术为导向的因素可能会暴露成败问题。
We believe that the development team has a responsibility to explain the consequences of not addressing these nonfunctional or cross-functional requirements. We’re really all part of one product team that wants to deliver good value, and these technology-oriented factors might expose make-or-break issues.
许多这些非功能性和跨功能性问题对于许多应用程序来说被认为是低风险的,因此不会添加到测试计划中。但是,当您计划项目时,您应该考虑每个领域的风险,在您的测试计划中解决它们,并将测试它们所需的工具和资源包括在您的项目计划中。
Many of these nonfunctional and cross-functional issues are deemed low-risk for many applications and so are not added to the test plan. However, when you are planning your project, you should think about the risks in each of these areas, address them in your test plan, and include the tools and resources needed for testing them in your project plan.
由于此象限中的测试类型多种多样,因此我们将提供可能对我们有用的工具示例,而不是单独的工具包部分。工具,无论是自产的还是收购的,都是第 4 象限测试工作取得成功所必需的。尽管如此,做工作的人还是很重要的,所以让我们考虑一下敏捷团队中的哪些人可以执行这些测试。
Because the types of testing in this quadrant are so diverse, we’ll give examples of tools that might be helpful as we go along instead of a separate toolkit section. Tools, whether homegrown or acquired, are essential to succeed with Quadrant 4 testing efforts. Still, the people doing the work count, so let’s consider who on an agile team can perform these tests.
所有关于敏捷的文献都在谈论团队是多面手;任何人都应该能够选择一项任务并完成它。我们知道这并不总是实用的,但我们的想法是能够分享知识,这样人们就不会成为信息孤岛。
All of the agile literature talks about teams being generalists; anyone should be able to pick up a task and do it. We know that isn’t always practical, but the idea is to be able to share the knowledge so that people don’t become silos of information.
但是,有许多任务需要专业知识。一个很好的例子是安全测试。我们不是在谈论应用程序内的安全性,例如谁有权管理它。因为这种类型的安全性实际上是功能需求的一部分,并且会被常规故事所涵盖,所以验证它是否有效属于前三个象限。我们正在谈论探测外部安全漏洞并了解黑客利用的系统中的漏洞类型。那是一套专门的技能。
However, there are many tasks that need specialized knowledge. A good example is security testing. We’re not talking about security within an application, such as who has access rights to administer it. Because that type of security is really part of the functional requirements and will be covered by regular stories, verifying that it works falls within the first three quadrants. We’re talking about probing for external security flaws and knowing the types of vulnerabilities in systems that hackers exploit. That is a specialized skill set.
性能测试可以由测试人员和程序员合作完成,并为他们的特定需求构建简单的工具。一些组织购买负载测试工具,需要专门研究该工具的团队成员来构建脚本并分析和解释结果。软件开发组织(尤其是小型软件开发组织)可能很难拥有足够的资源来为测试复制准确的生产级负载,因此可能需要外部性能测试提供商。
Performance testing can be done by testers and programmers collaborating and building simple tools for their specific needs. Some organizations purchase load-testing tools that require team members who specialize in that tool to build the scripts and analyze and interpret the results. It can be difficult for a software development organization, especially a small one, to have enough resources to duplicate an accurate production-level load for a test, so external providers of performance testing may be needed.
较大的组织可能有一些小组,例如您的团队可以用来帮助进行数据转换的数据库专家,可以帮助您识别应用程序风险的安全小组,或者可以帮助您测试恢复或故障转移的生产支持团队。与这些专家建立密切的关系。你需要作为一个虚拟团队一起工作,以收集你需要的关于你的产品的信息。
Larger organizations may have groups such as database experts that your team can use to help with data conversion, security groups that will help you identify risks to your application, or a production support team that can help you test recovery or failover. Build a close relationship with these specialists. You’ll need to work together as a virtual team to gather the information you need about your product.
第 15 章,“发布或主题规划中的测试人员活动”,解释了如何规划与外部团队的合作。
Chapter 15, “Tester Activities in Release or Theme Planning,” explains how to plan to work with external teams.
您团队中的技能组合越多样化,您就越不可能需要外部顾问来帮助您。确定每个项目所需的资源。许多团队发现优秀的技术测试员或工具匠可以承担其中的许多任务。如果团队中已有人可以学习所需的任何专业知识,那就太好了;否则,请引入您需要的专业知识。
The more diverse the skill sets are in your team, the less likely you are to need outside consultants to help you. Identify the resources you need for each project. Many teams find that a good technical tester or toolsmith can take on many of these tasks. If someone already on the team can learn whatever specialized knowledge is required, great; otherwise, bring in the expertise you need.
无论您的团队是否为这些类型的测试引入额外资源,您的团队仍然有责任确保完成最少的测试。这些测试提供的信息可能会在某些领域产生新的故事和任务,例如更改架构以获得更好的可扩展性或实施系统范围的安全解决方案。确保完成从批评产品的测试到推动将改进产品非功能方面的更改的测试的反馈循环。
Regardless of whether or not your team brings in additional resources for these types of tests, your team is still responsible for making sure the minimum testing is done. The information these tests provide may result in new stories and tasks in areas such as changing the architecture for better scalability or implementing a system-wide security solution. Be sure to complete the feedback loop from tests that critique the product to tests that drive changes that will improve the nonfunctional aspects of the product.
仅仅因为这是四个敏捷测试象限中的第四个并不意味着这些测试排在最后。您的团队需要考虑何时进行性能、安全和“功能”测试,以确保您的产品提供正确的商业价值。
Just because this is the fourth out of four agile testing quadrants doesn’t mean these tests come last. Your team needs to think about when to do performance, security, and “ility” tests so that you ensure your product delivers the right business value.
与功能测试一样,支持团队的面向技术的测试越早完成,修复发现的任何问题的成本就越低。然而,许多跨功能测试的成本很高,而且很难在小块中进行。
As with functional testing, the sooner technology-facing tests that support the team are completed, the cheaper it is to fix any issues that are found. However, many of the cross-functional tests are expensive and hard to do in small chunks.
可以编写技术故事来解决特定需求,例如:“作为用户 Abby,我需要在 20 秒内检索报告 X,以便我可以快速做出决定。” 这个故事是关于性能的,需要编写专门的测试,它可以与故事一起完成以对报告进行编码,或者在以后的迭代中完成。
Technical stories can be written to address specific requirements, such as: “As user Abby, I need to retrieve report X in less than 20 seconds so that I can make a decision quickly.” This story is about performance and requires specialized tests to be written, and it can be done along with the story to code the report, or in a later iteration.
考虑在你的故事板上单独一行来处理整个产品所需的任务。Lisa 的团队使用这个区域来放置诸如“评估负载测试工具”或“建立性能测试基线”之类的卡片。珍妮特成功地使用了不同颜色的卡片来表明这个故事是针对从组织其他领域借来的专家角色之一。
Consider a separate row on your story board for tasks needed by the product as a whole. Lisa’s team uses this area to put cards such as “Evaluate load test tools” or “Establish a performance test baseline.” Janet has successfully used different colored cards to show that the story is meant for one of the expert roles borrowed from other areas of the organization.
如果您正在尝试为完整的端到端工作流建立基线,则某些性能测试可能需要等到大部分应用程序构建完成。如果性能和可靠性是重中之重,您需要找到一种方法来在项目早期测试它们。确定故事的优先级,以便钢丝或薄片尽早完成。您应该能够创建一个可以运行的性能测试,并在您向工作流中添加越来越多的功能时继续运行。这可能使您能够及早发现性能问题并重新设计系统架构以进行改进。对于许多应用程序,如果没有必要的性能,正确的功能是无关紧要的。
Some performance tests might need to wait until much of the application is built if you are trying to baseline full end-to-end workflows. If performance and reliability are a top priority, you need to find a way to test those early in the project. Prioritize stories so that a steel thread or thin slice is complete early. You should be able to create a performance test that can be run and continue to run as you add more and more functionality to the workflow. This may enable you to catch performance issues early and redesign the system architecture for improvements. For many applications, correct functionality is irrelevant without the necessary performance.
考虑非功能性测试的时间是在发布或主题规划期间。计划尽早开始,根据需要处理小的增量。对于每次迭代,查看您的团队需要执行哪些任务,以确定代码设计是否可靠、可扩展、可用且安全。在下一节中,我们将研究一些不同类型的象限 4 测试。
The time to think about your nonfunctional tests is during release or theme planning. Plan to start early, tackling small increments as needed. For each iteration, see what tasks your team needs in order to determine whether the code design is reliable, scalable, usable, and secure. In the next section, we’ll look at some different types of Quadrant 4 tests.
如果我们可以只关注应用程序所需的行为和功能,生活就会变得如此简单。不幸的是,我们必须关注安全性、可维护性、互操作性、兼容性、可靠性和可安装性等质量。让我们来看看其中的一些“能力”。
If we could just focus on the desired behavior and functionality of the application, life would be so simple. Unfortunately, we have to be concerned with qualities such as security, maintainability, interoperability, compatibility, reliability, and installability. Let’s take a look at some of these “ilities.”
好的,它并没有以 -ility 结尾,但我们将它包含在“ility”桶中,因为我们使用面向技术的测试来评估产品的安全方面。如今,安全是每个组织的首要任务。每个组织都需要确保其软件的机密性和完整性。他们想要验证诸如不可否认之类的概念,即保证消息已由声称已发送消息的一方发送并由声称已收到消息的一方接收。应用程序需要执行正确的身份验证,确认每个用户的身份和授权,以便只允许用户访问他们有权使用的服务。测试这么多不同的安全方面并不容易。
OK, it doesn’t end in -ility, but we include it in the “ility” bucket because we use technology-facing tests to appraise the security aspects of the product. Security is a top priority for every organization these days. Every organization needs to ensure the confidentiality and integrity of their software. They want to verify concepts such as no repudiation, a guarantee that the message has been sent by the party that claims to have sent it and received by the party that claims to have received it. The application needs to perform the correct authentication, confirming each user’s identity, and authorization, in order to allow the user access only to the services they’re authorized to use. Testing so many different aspects of security isn’t easy.
在急于交付功能时,新成立的组织中的业务专家和开发团队可能不会首先考虑安全性。他们只是想让一些软件工作,这样他们就可以做生意了。授权通常是安全测试的唯一方面,他们将其视为业务功能的一部分。
In the rush to deliver functionality, both business experts and development teams in newly started organizations may not be thinking of security first. They just want to get some software working so they can do business. Authorization is often the only aspect of security testing that they consider as part of business functionality.
对此类工作进行预算必须是业务优先事项。根据贵公司的优先事项和资源,有多种选择可供选择。在投入大量时间和精力之前了解您的需求和风险。
Budgeting this type of work has to be a business priority. There’s a range of alternatives available, depending on your company’s priorities and resources. Understand your needs and the risks before you invest a lot of time and energy.
擅长安全测试的测试人员可以执行基于安全风险的测试,这是通过分析架构风险、攻击模式或滥用和误用案例来驱动的。当需要专业技能时,带来你需要的东西,但团队仍然负责确保测试完成。
Testers who are skilled in security testing can perform security risk-based testing, which is driven by analyzing the architectural risk, attack patterns, or abuse and misuse cases. When specialized skills are required, bring in what you need, but the team is still responsible for making sure the testing gets done.
有多种自动化工具可以帮助进行安全验证。静态分析工具可以在不执行应用程序的情况下检查代码,可以检测代码中可能多年不会出现的潜在安全漏洞。实时运行的动态分析工具可以测试 SQL 注入和跨站点脚本等漏洞。知识渊博的安全测试人员进行的手动探索性测试对于检测自动化测试可能遗漏的问题是必不可少的。
There are a variety of automated tools to help with security verification. Static analysis tools, which can examine the code without executing the application, can detect potential security flaws in the code that might not otherwise show up for years. Dynamic analysis tools, which run in real time, can test for vulnerabilities such as SQL injection and cross-site scripting. Manual exploratory testing by a knowledgeable security tester is indispensable to detect issues that automated tests can miss.
仅此对安全测试的简要介绍就说明了为什么专门的培训和工具对于做好它如此重要。对于大多数组织而言,此测试是绝对必需的。一次安全入侵可能足以让一家公司倒闭。即使概率很低,赌注也太高了,不能推迟这些测试。
Just this brief look at security testing shows why specialized training and tools are so important to do a good job of it. For most organizations, this testing is absolutely required. One security intrusion might be enough to take a company out of business. Even if the probability were low, the stakes are too high to put off these tests.
维护成本高昂的代码可能不会像安全漏洞那样迅速扼杀组织的盈利能力,但它可能导致长期、缓慢的死亡。在下一节中,我们将考虑验证可维护性的方法。
Code that costs a lot to maintain might not kill an organization’s profitability as quickly as a security breach, but it could lead to a long, slow death. In the next section we consider ways to verify maintainability.
可维护性不是很容易测试的东西。在传统项目中,通常通过使用完整的代码审查或检查来完成。敏捷团队经常使用结对编程,它具有内置的持续代码审查。还有其他方法可以确保代码和测试保持可维护性。
Maintainability is not something that is easy to test. In traditional projects, it’s often done by the use of full code reviews or inspections. Agile teams often use pair programming, which has built-in continual code review. There are other ways to make sure the code and tests stay maintainable.
我们鼓励开发团队制定他们遵循的应用程序代码、测试框架和测试本身的标准和指南。制定自己的标准而不是让其他独立团队制定标准的团队将更有可能遵循这些标准,因为它们对他们有意义。
We encourage development teams to develop standards and guidelines that they follow for application code, the test frameworks, and the tests themselves. Teams that develop their own standards, rather than having them set by some other independent team, will be more likely to follow them because they make sense to them.
我们所说的标准种类包括方法名称或测试名称的命名约定。所有指南都应该易于遵循,并使可维护性更容易。示例是:“成功始终为零,失败必须为负值”、“每个类或模块应该只有一个单一职责”或“所有函数必须是单入口、单出口”。
The kinds of standards we mean include naming conventions for method names or test names. All guidelines should be simple to follow and make maintainability easier. Examples are: “Success is always zero and failure must be a negative value,” “Each class or module should have only one single responsibility,” or “All functions must be single entry, single exit.”
开发 GUI 的标准还使应用程序更易于测试和维护,因为测试人员知道会发生什么并且不需要怀疑行为是对还是错。如果您从 GUI 自动执行测试,它还会增加可测试性。简单的标准,例如“为所有 GUI 对象使用名称而不是默认为计算机分配的标识符”或“页面上不能有两个具有相同名称的字段”帮助团队达到代码可维护的水平,因为为其提供覆盖的自动化测试。
Standards for developing the GUI also make the application more testable and maintainable, because testers know what to expect and don’t need to wonder whether a behavior is right or wrong. It also adds to testability if you are automating tests from the GUI. Simple standards such as, “Use names for all GUI objects rather than defaulting to the computer assigned identifier” or “You cannot have two fields with the same name on a page” help the team achieve a level where the code is maintainable, as are the automated tests that provide coverage for it.
可维护代码支持共享代码所有权。如果所有代码都以相同的风格编写并且团队中的每个人都容易理解,那么程序员从一个领域转移到另一个领域要容易得多。复杂性增加了风险,也使代码更难理解。简单性的 XP 值应该应用于代码。简单的编码标准还可以包括诸如“避免重复——不要复制粘贴方法”之类的指南。这些相同的概念适用于测试框架和测试本身。
Maintainable code supports shared code ownership. It is much easier for a programmer to move from one area to another if all code is written in the same style and easily understood by everyone on the team. Complexity adds risk and also makes code harder to understand. The XP value of simplicity should be applied to code. Simple coding standards can also include guidelines such as, “Avoid duplication—Don’t copy-paste methods.” These same concepts apply to test frameworks and the tests themselves.
可维护性也是自动化测试的一个重要因素。测试工具在使它们易于使用的特性方面落后于编程工具maintain,比如IDE插件,让编写和维护测试脚本更简单、更高效。这种情况变化很快,因此寻找可提供轻松重构和搜索与替换的工具,以及可轻松修改脚本的其他实用程序。
Maintainability is an important factor for automated tests as well. Test tools have lagged behind programming tools in features that make them easy to maintain, such as IDE plug-ins to make writing and maintaining test scripts simpler and more efficient. That’s changing fast, so look for tools that provide easy refactoring and search-and-replace, and for other utilities that make it easy to modify the scripts.
数据库的可维护性也很重要。数据库设计需要灵活和可用。每次迭代都可能带来任务来添加或删除表、列、约束或触发器,或者进行某种数据转换。如果数据库设计不佳或数据库中充斥着无效数据,这些任务就会成为瓶颈。
Database maintainability is also important. The database design needs to be flexible and usable. Every iteration might bring tasks to add or remove tables, columns, constraints, or triggers, or to do some kind of data conversion. These tasks become a bottleneck if the database design is poor or the database is cluttered with invalid data.
计划时间来评估数据库对团队速度的影响,并像处理生产和测试代码一样重构它。应用程序、测试和执行环境的所有方面的可维护性更多是评估和重构的问题,而不是直接测试。如果你的速度在下降,是因为部分代码难以处理,还是因为数据库难以修改?
Plan time to evaluate the database’s impact on team velocity, and refactor it just as you do production and test code. Maintainability of all aspects of the application, test, and execution environments is more a matter of assessment and refactoring than direct testing. If your velocity is going down, is it because parts of the code are hard to work on, or is it that the database is difficult to modify?
互操作性是指不同系统和组织协同工作和共享信息的能力。互操作性测试着眼于两个或多个通信系统之间的端到端功能。这些测试是在用户(人或软件应用程序)的上下文中完成的,并查看功能行为。
Interoperability refers to the capability of diverse systems and organizations to work together and share information. Interoperability testing looks at end-to-end functionality between two or more communicating systems. These tests are done in the context of the user—human or a software application—and look at functional behavior.
在敏捷开发中,互操作性测试可以在开发周期的早期完成。我们在每次迭代结束时都有一个可运行的可部署系统,以便我们可以部署和设置与其他系统的测试。
In agile development, interoperability testing can be done early in the development cycle. We have a working, deployable system at the end of each iteration so that we can deploy and set up testing with other systems.
象限 1 包括代码集成测试,它们是组件之间的测试,但在企业系统中还有另一个级别的集成测试。您可能会发现自己通过开放或专有接口集成系统。您为您的系统开发的 API 可能使您的用户能够轻松地设置一个框架,以便他们轻松地进行测试。为您的客户进行更轻松的测试可以更快地接受。
Quadrant 1 includes code integration tests, which are tests between components, but there is a whole other level of integration tests in enterprise systems. You might find yourself integrating systems through open or proprietary interfaces. The API you develop for your system might enable your users to easily set up a framework for them to test easily. Easier testing for your customer makes for faster acceptance.
在第 20 章“成功交付”中,我们将更多地讨论这一级别测试的重要性。
In Chapter 20, “Successful Delivery,” we discuss more about the importance of this level of testing.
在 Janet 参与的一个项目中,在客户现场设置了测试系统,以便他们可以尽早开始将它们与自己的系统集成。根据需要更改现有系统的接口,并在每次新部署时进行测试。
In one project Janet worked on, test systems were set up at the customer’s site so that they could start to integrate them with their own systems early. Interfaces to existing systems were changed as needed and tested with each new deployment.
如果你的团队工作的系统必须与外部系统一起工作,你可能无法在你的测试环境中代表它们,除非使用模拟其他系统或设备行为的存根和驱动程序。在这种情况下,开发完成后进行测试可能是不可避免的。您可能必须在多个团队共享的测试环境中安排测试时间。
If the system your team works on has to work together with external systems, you may not be able to represent them all in your test environments except with stubs and drivers that simulate the behavior of the other systems or equipment. This is one situation where testing after development is complete might be unavoidable. You might have to schedule test time in a test environment shared by several teams.
考虑您需要与之通信的所有系统,并确保您提前计划拥有一个合适的环境来一起测试它们。您还需要规划资源来测试您的应用程序是否与可能使用它的各种操作系统、浏览器、客户端、服务器和硬件兼容。接下来我们将讨论兼容性测试。
Consider all of the systems with which yours needs to communicate, and make sure you plan ahead to have an appropriate environment for testing them together. You’ll also need to plan resources for testing that your application is compatible with the various operating systems, browsers, clients, servers, and hardware with which it might be used. We’ll discuss compatibility testing next.
您从事的项目类型决定了需要进行多少兼容性测试。如果您有一个 Web 应用程序并且您的客户遍布全球,您将需要考虑所有类型的浏览器和操作系统。如果您要交付自定义企业应用程序,您可能会减少兼容性测试的数量,因为您可能能够决定支持哪些版本。
The type of project you’re working on dictates how much compatibility testing is required. If you have a web application and your customers are worldwide, you will need to think about all types of browsers and operating systems. If you are delivering a custom enterprise application, you can probably reduce the amount of compatibility testing, because you might be able to dictate which versions are supported.
由于每个新屏幕都是作为用户界面故事的一部分开发的,因此最好检查其在所有支持的浏览器中的可操作性。可以将一个简单的任务添加到故事中以在所有浏览器上进行测试。
As each new screen is developed as part of a user interface story, it is a good idea to check its operability in all supported browsers. A simple task can be added to the story to test on all browsers.
珍妮特工作的一个组织必须测试与视障人士阅读软件的兼容性。虽然公司没有正式测试实验室,它在团队区域附近提供测试机器,以便于访问。测试人员定期检查以确保新功能仍然与第三方工具兼容。很容易修复在开发早期发现的问题。
One organization that Janet worked at had to test compatibility with reading software for the visual impaired. Although the company had no formal test lab, it had test machines available near the team area for easy access. The testers made periodic checks to make sure that new functionality was still compatible with the third-party tools. It was easy to fix problems that were discovered early during development.
让测试机器可用于不同的操作系统或浏览器或第三方应用程序,这些应用程序需要与被测系统一起工作,这使测试人员更容易确保与每个新故事或在迭代结束时的兼容性。当您开始一个新的主题或项目时,请考虑您可能需要的资源来验证兼容性。如果您开始开发一种全新的产品,您可能需要为它建立一个测试实验室。确保您的团队获得有关最终用户的硬件、操作系统、浏览器及其版本的信息。如果新浏览器版本的使用百分比增长到足够大,可能是时候开始将该版本纳入兼容性测试了。
Having test machines available with different operating systems or browsers or third-party applications that need to work with the system under test makes it easier for the testers to ensure compatibility with each new story or at the end of an iteration. When you start a new theme or project, think about the resources you might need to verify compatibility. If you’re starting on a brand new product, you might have to build up a test lab for it. Make sure your team gets information on your end users’ hardware, operating systems, browsers, and versions of each. If the percentage of use of a new browser version has grown large enough, it might be time to start including that version in your compatibility testing.
当您选择或创建功能测试工具时,请确保有一种简单的方法可以在不同版本的浏览器、操作系统和硬件上运行相同的脚本。例如,Lisa 的团队可以在 Windows、Solaris 和 Linux 上运行的每台服务器上使用相同的 GUI 回归测试套件。功能测试脚本也可用于可靠性测试。让我们接下来看看。
When you select or create functional test tools, make sure there’s an easy way to run the same script with different versions of browsers, operating systems, and hardware. For example, Lisa’s team could use the same suite of GUI regression tests on each of the servers running on Windows, Solaris, and Linux. Functional test scripts can also be used for reliability testing. Let’s look at that next.
软件的可靠性可以指系统在常规情况和意外情况下执行和维护其功能的能力。该系统还必须以一致性和可重复性执行和维护其功能。可靠性分析回答了“它会运行多长时间才会坏掉?”这个问题。用于衡量可靠性的一些统计数据是:
Reliability of software can be referred to as the ability of a system to perform and maintain its functions in routine circumstances as well as unexpected circumstances. The system also must perform and maintain its functions with consistency and repeatability. Reliability analysis answers the question, “How long will it run before it breaks?” Some statistics used to measure reliability are:
平均无故障时间:从初始操作到第一次出现故障或故障之间的平均时间。换句话说,系统在第一次出现故障之前可以运行多长时间?
Mean time to failure: The average or mean time between initial operation and the first occurrence of a failure or malfunction. In other words, how long can the system run before it fails the first time?
Mean time between failures(平均故障间隔时间):一种可靠性的统计量度,计算得出预期的平均故障间隔时间。越长越好。
Mean time between failures: A statistical measure of reliability, this is calculated to indicate the anticipated average time between failures. The longer the better.
在传统项目中,我们过去常常安排数周的可靠性测试,试图运行与日常工作相匹配的模拟。现在,我们应该能够在每次迭代结束时交付,那么我们如何安排可靠性测试呢?
In traditional projects, we used to schedule weeks of reliability testing that tried to run simulations that matched a regular day’s work. Now, we should be able to deliver at the end of every iteration, so how can we schedule reliability tests?
我们有定期运行的自动化单元和验收测试。要进行可靠性测试,我们只需要使用相同的测试并一遍又一遍地运行它们。理想情况下,您将使用收集的显示日常使用情况的统计数据,创建一个反映使用情况的脚本,并在您的团队认为足以证明稳定性的稳定构建上运行它。您可以将随机数据输入到测试中以模拟生产使用,并确保应用程序不会因无效输入而崩溃。当然,您可能希望反映高峰使用情况以确保它也能处理繁忙时间。
We have automated unit and acceptance tests running on a regular basis. To do a reliability test, we simply need to use those same tests and run them over and over. Ideally, you would use statistics gathered that show daily usage, create a script that mirrors the usage, and run it on a stable build for however long your team thinks is adequate to prove stability. You can input random data into the tests to simulate production use and make sure the application doesn’t crash because of invalid inputs. Of course, you might want to mirror peak usage to make sure that it handles busy times as well.
您可以在每次迭代中创建故事来开发这些脚本并在将新功能添加到应用程序时添加新功能。您的验收测试可以非常具体,例如“功能 X 必须在 24 小时内至少执行 10,000 次操作,持续至少 3 天。”
You can create stories in each iteration to develop these scripts and add new functionality as it is added to the application. Your acceptance tests could be very specific such as, “Functionality X must perform 10,000 operations in a 24-hour period for a minimum of 3 days.”
当心:运行一千次测试没有任何严重问题并不意味着您拥有可靠的软件。您必须运行正确的测试。要使可靠性测试有效,请考虑您的应用程序以及它在一段时间内全天、每天、如何使用。指定旨在证明您的应用程序即使在高峰时段也能够满足客户需求的测试。
Beware: Running a thousand tests without any serious problems doesn’t mean you have reliable software. You have to run the right tests. To make a reliability test effective, think about your application and how it is used all day, every day, over a period of time. Specify tests that are aimed at demonstrating that your application will be able to meet your customers’ needs, even during peak times.
以可衡量目标的形式向客户团队询问他们的可靠性标准。例如,如果每 10,000 笔交易出现 10 个或更少的错误,或者 Web 应用程序在 99.999% 的时间内可用,他们可能会认为系统是可靠的。从停电和其他灾难中恢复可能是可靠性目标的一部分,并将以服务水平协议的形式说明。知道它们是什么。一些行业有自己的软件可靠性标准和指南。
Ask the customer team for their reliability criteria in the form of measurable goals. For example, they might consider the system reliable if ten or fewer errors occur for every 10,000 transactions, or the web application is available 99.999% of the time. Recovery from power outages and other disasters might be part of the reliability objectives, and will be stated in the form of Service Level Agreements. Know what they are. Some industries have their own software reliability standards and guidelines.
使用正确的程序员和客户测试来推动开发应该会增强应用程序的可靠性,因为这通常会带来更好的设计和更少的缺陷。根据需要编写额外的故事和任务,以交付满足组织可靠性标准的系统。
Driving development with the right programmer and customer tests should enhance the application’s reliability, because this usually leads to better design and fewer defects. Write additional stories and tasks as needed to deliver a system that meets the organization’s reliability standards.
您的产品在启动和运行后可能是可靠的,但它也需要在所有受支持的环境中可供所有用户安装。这是遵循敏捷原则给我们带来优势的另一个领域。
Your product might be reliable after it’s up and running, but it also needs to be installable by all users, in all supported environments. This is another area where following agile principles gives us an advantage.
成功的敏捷团队的基石之一是持续集成。这意味着构建可以在一天中的任何时间进行测试。许多团队选择每天将一个或多个成功的构建部署到测试环境中。
One of the cornerstones of a successful agile team is continuous integration. This means that a build is ready for testing anytime during the day. Many teams choose to deploy one or more of the successful builds into test environments on a daily basis.
自动化部署创建可重复性并使部署成为非事件。这让我们很兴奋,因为我们已经经历了数周的时间来尝试集成和安装一个新系统。我们知道,如果我们构建一次并将相同的构建部署到多个环境,我们就已经形成了一致性。
Automating the deployment creates repeatability and makes deployment a non-event. This is exciting to us because we have experienced weeks of trying to integrate and install a new system. We know that if we build once and deploy the same build to multiple environments, we have developed consistency.
与任何其他功能一样,需要评估与安装相关的风险并相应地确定测试量。我们的建议是尽早并经常这样做,并尽可能自动化该过程。
As with any other functionality, risks associated with installation need to be evaluated and the amount of testing determined accordingly. Our advice is to do it early and often, and automate the process if possible.
Chapter 20, “Successful Delivery,” has more on installation testing.
根据您的产品领域,还有其他“能力”需要测试。安全关键软件,例如用于医疗设备和飞机控制系统的软件,需要进行广泛的安全测试,而回归测试可能会包含与安全相关的测试。系统冗余和故障转移测试对于此类产品尤为重要。您的团队可能需要查看有关软件相关安全问题的行业数据并使用额外的代码审查。可配置性、可审核性、可移植性、健壮性和可扩展性只是您的团队可能需要通过面向技术的测试来评估的几个品质。
There are other “ilities” to test, depending on your product’s domain. Safety-critical software, such as that used in medical devices and aircraft control systems, requires extensive safety testing, and the regression tests probably would contain tests related to safety. System redundancy and failover tests would be especially important for such a product. Your team might need to look at industry data around software-related safety issues and use extra code reviews. Configurability, auditability, portability, robustness, and extensibility are just a few of the qualities your team might need to evaluate with technology-facing tests.
无论您需要测试什么“能力”,请使用增量方法。首先引出客户团队的要求和他们针对特定质量领域的目标示例。编写面向业务的测试以确保代码旨在满足这些目标。在第一次迭代中,团队可能会做一些研究并提出测试策略来评估产品的现有质量水平。下一步可能是创建一个合适的测试环境,研究工具,或者从一些手动测试开始。
Whatever “ility” you need to test, use an incremental approach. Start by eliciting the customer team’s requirements and examples of their objectives for that particular area of quality. Write business-facing tests to make sure the code is designed to meet those goals. In the first iteration, the team might do some research and come up with a test strategy to evaluate the existing quality level of the product. The next step might be to create a suitable test environment, to research tools, or to start with some manual tests.
当您了解应用程序如何满足客户的要求时,使用新的象限 1 和 2 测试关闭循环,使应用程序更接近该特定属性的目标。还建议对下一节中介绍的性能、负载和其他测试使用增量方法。
As you learn how the application measures up to the customers’ requirements, close the loop with new Quadrant 1 and 2 tests that drive the application closer to the goals for that particular property. An incremental approach is also recommended for performance, load, and other tests that are addressed in the next section.
性能、负载、压力和可伸缩性测试都属于第 4 象限,因为它们的技术重点。通常需要专业技能,尽管许多团队已经想出了在这些领域进行自己测试的方法。我们先谈谈可扩展性,因为它经常被遗忘。
Performance, load, stress, and scalability testing all fall into Quadrant 4 because of their technology focus. Often specialized skills are required, although many teams have figured out ways to do their own testing in these areas. Let’s talk about scalability first, because it is often forgotten.
可扩展性测试验证应用程序在添加更多用户时仍然可靠。这真正的意思是,“您的系统能否处理不断增长的客户群的容量?” 听上去很简单,其实不然,是一个敏捷团队通常无法独自解决的问题。
Scalability testing verifies the application remains reliable when more users are added. What that really means is, “Can your system handle the capacity of a growing customer base?” It sounds simple, but really isn’t, and is a problem that an agile team usually can’t solve by itself.
重要的是要考虑整个系统,而不仅仅是应用程序本身。例如,网络往往是瓶颈,因为它无法处理增加的吞吐量。数据库呢?它会扩展吗?您正在使用的硬件会处理正在考虑的新负载吗?只是添加新硬件很简单,还是瓶颈?
It is important to think about the whole system and not just the application itself. For example, the network is often the bottleneck, because it can’t handle the increased throughput. What about the database? Will it scale? Will the hardware you are using handle the new loads being considered? Is it simple just to add new hardware, or is it the bottleneck?
您需要走出团队才能获得解决可扩展性问题所需的答案,因此请提前计划。
You will need to go outside the team to get the answers you require to address scalability issues, so plan ahead.
性能测试通常用于帮助识别系统中的瓶颈或为将来的测试建立基线。这样做也是为了确保符合性能目标和要求,并帮助利益相关者做出与被测试应用程序的整体质量相关的明智决策。
Performance testing is usually done to help identify bottlenecks in a system or to establish a baseline for future testing. It is also done to ensure compliance with performance goals and requirements, and to help stakeholders make informed decisions related to the overall quality of the application being tested.
随着越来越多的用户同时访问系统,负载测试会评估系统行为。压力测试评估应用程序在高于预期的负载下的健壮性。应用程序会随着业务的增长而扩展吗?对于某些应用程序,响应时间等特性可能比功能更为重要。
Load testing evaluates system behavior as more and more users access the system at the same time. Stress testing evaluates the robustness of the application under higher-than-expected loads. Will the application scale as the business grows? Characteristics such as response time can be more critical than functionality for some applications.
Grig Gheorghiu [ 2005 ] 强调需要明确定义的期望才能从性能测试中获得价值。他说,“如果你不知道在系统方面你想去哪里,那么你选择哪个方向就无关紧要了(还记得爱丽丝和柴郡猫吗?)。” 例如,您可能想知道 Web 应用程序的并发用户数和可接受的响应时间。
Grig Gheorghiu [2005] emphasizes the need for clearly defined expectations to get value from performance testing. He says, “If you don’t know where you want to go in terms of the system, then it matters little which direction you take (remember Alice and the Cheshire Cat?).” For example, you probably want to know the number of concurrent users and the acceptable response time for a web application.
定义性能目标后,您可以使用各种工具对系统施加负载并检查瓶颈。这可以在单元级别完成,使用 JUnitPerf、httperf 或自制工具等工具。Apache JMeter、The Grinder、Pounder、ftptt 和 OpenWebLoad 是撰写本文时可用的众多开源性能和负载测试工具的更多示例。其中一些,例如 JMeter,可用于多种服务器类型,从SOAP到 LDAP 再到 POP3 邮件。许多商业工具选项也可用,包括 NeoLoad、WebLoad、eValid LoadTest、LoadRunner 和 SOATest。
After you’ve defined your performance goals, you can use a variety of tools to put a load on the system and check for bottlenecks. This can be done at the unit level, with tools such as JUnitPerf, httperf, or a home-grown harness. Apache JMeter, The Grinder, Pounder, ftptt, and OpenWebLoad are more examples of the many open source performance and load test tools available at the time of this writing. Some of these, such as JMeter, can be used on a variety of server types, from SOAP to LDAP to POP3 mail. Plenty of commercial tool options are available too, including NeoLoad, WebLoad, eValid LoadTest, LoadRunner, and SOATest.
使用这些工具来查找性能瓶颈。Lisa 的团队使用 JProfiler 查找应用程序瓶颈和内存泄漏,并使用 JConsole 分析数据库使用情况。.NET 和其他环境存在类似的工具,包括 .NET Memory Profiler 和 ANTS Profiler Pro。正如 Grig 指出的那样,有特定于数据库的分析器可以查明数据库级别的性能问题;请您的数据库专家与您合作。您的系统管理员可以帮助您使用 shell 命令(例如 top)或工具(例如 PerfMon)来监控 CPU、内存、交换区、磁盘 I/O 和其他硬件资源。类似的工具在网络级别可用,例如 NetScout。
Use these tools to look for performance bottlenecks. Lisa’s team uses JProfiler to look for application bottlenecks and memory leaks, and JConsole to analyze database usage. Similar tools exist for .NET and other environments, including .NET Memory Profiler and ANTS Profiler Pro. As Grig points out, there are database-specific profilers to pinpoint performance issues at the database level; ask your database experts to work with you. Your system administrators can help you use shell commands such as top, or tools such as PerfMon to monitor CPU, memory, swap, disk I/O, and other hardware resources. Similar tools are available at the network level, for example, NetScout.
您还可以使用团队最熟悉的工具。在一个项目中,Janet 与其中一位程序员密切合作来创建测试。她帮助他根据客户的性能和负载期望定义所需的测试,然后他使用 JUnit 将这些测试自动化。他们一起分析结果并向客户报告。
You can also use the tools the team is most familiar with. In one project, Janet worked very closely with one of the programmers to create the tests. She helped him to define the tests needed based on customer’s performance and load expectations, and he automated them using JUnit. Together they analyzed the results to report back to the customer.
建立基线是评估绩效的良好开端。下一节将探讨性能测试的这一方面。
Establishing a baseline is a good first step for evaluating performance. The next section explores this aspect of performance testing.
性能调整可能会变成一个大项目,因此必须提供一个基准,您可以将其与新版本软件的性能进行比较。即使目前性能不是您最关心的问题,也不要忽视它。获得性能基线是个好主意,这样您以后就知道您的响应时间将朝哪个方向发展。Lisa 的公司托管了一个负载很小的网站。他们在站点上获得了负载测试基线,以便随着站点的增长,他们知道性能受到了怎样的影响。
Performance tuning can turn into a big project, so it is essential to provide a baseline that you can compare against new versions of the software on performance. Even if performance isn’t your biggest concern at the moment, don’t ignore it. It’s a good idea to get a performance baseline so that you know later which direction your response time is headed. Lisa’s company hosts a website that has had a small load on it. They got a load test baseline on the site so that as it grew, they’d know how performance was being affected.
如果已经为特定功能定义了特定的性能标准,我们建议将性能测试作为迭代的一部分进行,以确保在为时已晚而无法解决问题之前发现问题。
If there are specific performance criteria that have been defined for specific functionality, we suggest that performance testing be done as part of the iteration to ensure that issues are found before it is too late to fix them.
基准测试可以在发布期间的任何时间进行。如果添加了可能影响性能的新功能,例如复杂的查询,请重新运行测试以确保没有不利影响。这样,当开发团队仍然熟悉该功能时,您就有时间在周期的早期优化查询或代码。
Benchmarking can be done at any time during a release. If new functionality is added that might affect the performance, such as complicated queries, rerun the tests to make sure there are no adverse effects. This way, you have time to optimize the query or code early in the cycle when the development team is still familiar with the feature.
除非在模拟生产环境的环境中运行,否则任何性能、负载或压力测试都没有意义。让我们多谈谈环境。
Any performance, load, or stress test won’t be meaningful unless it’s run in an environment that mimics the production environment. Let’s talk more about environments.
性能测试的最终运行将帮助客户做出接受其产品的决定。为了获得准确的结果,需要在与生产设备相似的设备上运行测试。团队通常会使用较小的机器并推断结果以确定性能是否足以满足业务需求。报告测试结果时应明确指出这一点。
Final runs of the performance tests will help customers make decisions about accepting their product. For accurate results, tests need to be run on equipment that is similar to that of production. Often teams will use smaller machines and extrapolate the results to decide if the performance is sufficient for the business needs. This should be clearly noted when reporting test results.
也可以在发布期间的任何时间对应用程序施加压力以查看它在崩溃之前可以承受的负载,但通常客户不会将其视为高优先级,除非您的任务关键型系统具有大量负载。
Stressing the application to see what load it can take before it crashes can also be done anytime during the release, but usually it is not considered high-priority by customers unless you have a mission-critical system with lots of load.
受负载增加影响的一种资源是内存。在下一节中,我们将讨论内存管理。
One resource that is affected by increasing load is memory. In the next section, we discuss memory management.
内存通常根据用于 RAM、ROM、硬盘驱动器等的内存量(通常是最小值或最大值)来描述。您应该了解内存使用情况并注意是否存在泄漏,因为当应用程序在使用高峰期间处于生产状态时,它们可能会导致灾难性故障。一些编程语言更容易受到内存问题的影响,因此了解代码的优缺点将帮助您了解要注意什么。内存问题测试可以作为性能、负载和压力测试的一部分来完成。
Memory is usually described in terms of the amount (normally the minimum or maximum) of memory to be used for RAM, ROM, hard drives, and so on. You should be aware of memory usage and watch for leaks, because they can cause catastrophic failures when the application is in production during peak usage. Some programming languages are more susceptible to memory issues, so understanding the strengths and weaknesses of the code will assist you in knowing what to watch for. Testing for memory issues can be done as part of performance, load, and stress testing.
垃圾收集是一种用于将内存释放回程序的工具。但是,它可以掩盖严重的内存问题。如果您看到可用内存随着使用量稳定下降,然后突然增加到最大可用内存,您可能怀疑垃圾收集已经启动。观察模式中的异常情况或系统是否在大量使用下开始变慢。您可能需要监视一段时间并与程序员一起查找问题。修复可能很简单,例如更频繁地安排垃圾收集或将触发级别设置得更高。
Garbage collection is one tool used to release memory back to the program. However, it can mask severe memory issues. If you see the available memory steadily decreasing with usage and then all of a sudden increasing to maximum available, you might suspect the garbage collection has kicked in. Watch for anomalies in the pattern or whether the system starts to get slow under heavy usage. You may need to monitor for a while and work with the programmers to find the issue. The fix might be something simple, such as scheduling the garbage collection more often or setting the trigger level higher.
当您与程序员一起编写故事时,询问他们是否预计会出现记忆问题。如果您知道该地区可能存在风险,您可以专门进行测试。观察内存泄漏并不总是那么容易,但有一些工具可以提供帮助。这是程序员应该拥有易于使用的工具的领域。与他们合作以验证应用程序没有内存问题。执行上一节中描述的性能和负载测试以验证没有任何内存问题。
When you are working with the programmers on a story, ask them if they expect problems with memory. You can test specifically if you know there might be a risk in the area. Watching for memory leaks is not always easy, but there are tools to help. This is an area where programmers should have tools easily available. Collaborate with them to verify that the application is free of memory issues. Perform the performance and load tests described in the previous section to verify that there aren’t any memory problems.
您不必是如何进行面向技术的测试来批评产品以帮助您的团队计划和执行它的专家。您的团队可以从该象限评估需要进行哪些测试。在计划发布时谈论这些测试;如果您以前没有做过,您可以专门针对性能和负载创建一个测试计划。您将需要时间来获得所需的专业知识,方法是通过识别和学习技能来获得它,或者通过引入外部帮助来获得。与所有开发工作一样,将面向技术的测试分解为可以在每次迭代中解决和构建的小任务。
You don’t have to be an expert on how to do technology-facing testing that critiques the product to help your team plan for it and execute it. Your team can evaluate what tests it needs from this quadrant. Talk about these tests as you plan your release; you can create a test plan specifically for performance and load if you’ve not done it before. You will need time to obtain the expertise needed, either by acquiring it through identifying and learning the skills, or by bringing in outside help. As with all development efforts, break technology-facing tests into small tasks that can be addressed and built upon each iteration.
在本章中,我们探索了第四个敏捷测试象限,即评判产品的面向技术的测试。
In this chapter, we’ve explored the fourth agile testing quadrant, the technology-facing tests that critique the product.
开发团队应评估其是否拥有或能够获得进行这些测试的专业知识,或者是否需要计划引入外部资源。
The developer team should evaluate whether it has, or can acquire, the expertise to do these tests, or if it needs to plan to bring in external resources.
这些测试采用增量方法,在每次迭代中完成任务,确保有时间解决出现的任何问题并避免生产问题。
An incremental approach to these tests, completing tasks in each iteration, ensures time to address any issues that arise and avoid production problems.
团队应考虑各种类型的“能力”测试,包括安全性、可维护性、互操作性、兼容性、可靠性和可安装性测试,并应在适当的时间执行这些测试。
The team should consider various types of “ility” testing, including security, maintainability, interoperability, compatibility, reliability, and installability testing, and should execute these tests at appropriate times.
性能、可伸缩性、压力和负载测试应该从项目一开始就完成。
Performance, scalability, stress, and load testing should be done from the beginning of the project.
研究可能影响您的产品的内存管理问题,并计划测试以验证应用程序没有内存问题。
Research the memory management issues that might impact your product, and plan tests to verify the application is free of memory issues.
在第 6 章中,我们介绍了测试象限,在接下来的章节中,我们讨论了如何在敏捷项目中使用这些概念。在本章中,我们将通过一个使用来自所有四个象限的测试的敏捷团队的示例将它们整合在一起。
In Chapter 6, we introduced the testing quadrants, and in the chapters that followed we talked about how to use the concepts in your agile project. In this chapter, we’ll bring it all together with an example of an agile team that used tests from all four quadrants.
我们刚刚用五章讨论了每个象限(参见图 12-1)和可用于不同类型测试的工具示例。下一个技巧是了解您的项目需要哪些测试以及何时进行。在本章中,我们将带您了解一个敏捷项目的真实示例,该项目使用了来自所有四个敏捷测试象限的测试。
We’ve just spent five chapters talking about each of the quadrants (see Figure 12-1) and examples of tools you can use for the different types of testing. The next trick is to know which tests your project needs and when to do them. In this chapter, we’ll walk you through a real-life example of an agile project that used tests from all four agile testing quadrants.
下面的故事是关于一个组织使用各种本土和开源工具成功测试其整个系统的故事。Janet 与这个团队一起工作,Paul Rogers 是主要的测试架构师。这是保罗的故事。
The following story is about one organization’s success in testing its whole system using a variety of home-grown and open source tools. Janet worked with this team, and Paul Rogers was the primary test architect. This is Paul’s story.
该系统解决了远程油气生产井的监控问题。该解决方案结合了一个远程监控设备,该设备可以通过卫星通信信道使用专有协议传输数据并从中央监控站接收调整。
The system solves the problem of monitoring remote oil and gas production wells. The solution combines a remote monitoring device that can transmit data and receive adjustments from a central monitoring station using a proprietary protocol over a satellite communication channel.
远程数据监控系统架构如图12-2所示。油井上的测量设备,远程终端单元 (RTU),使用多种与测量设备通信的协议。来自每个 RTU 的数据通过卫星传输到位于客户总部的服务器。然后通过 Web 界面将其提供给用户。当特定读数超出正常操作限制时,可以通过电子邮件、传真或电话通知系统。Java消息服务 (JMS)提要和 Web 服务也可用于帮助与客户的其他应用程序集成。
Figure 12-2 shows the architecture of the Remote Data Monitoring system. The measurement devices on the oil wells, Remote Terminal Units (RTU), use a variety of protocols to communicate with the measurement device. This data from each RTU is transmitted via satellite to servers located at the client’s main office. It is then made available to users via a web interface. A notification system, via email, fax, or phone, is available when a particular reading is outside of normal operational limits. A Java Message Service (JMS) feed and web services are also available to help integration with clients’ other applications.
该软件应用程序是一个巨大的遗留系统,几乎没有单元测试。该团队正在使用新技术慢慢重建应用程序。
The software application was a huge legacy system that had few unit tests. The team was slowly rebuilding the application with new technology.
该团队由四名软件程序员、两名固件程序员、三到四名测试人员、一名产品工程师和一名场外经理组成。“真正的”客户在另一个国家。开发组使用XP实践,包括结对编程和 TDD。客户团队对积压工作使用缺陷跟踪系统,但故事的大部分可见性是通过索引卡。在迭代计划会议期间使用故事卡,任务板跟踪进度。
The team consisted of four software programmers, two firmware programmers, three to four testers, a product engineer, and an off-site manager. The “real” customer was in another country. The development team uses XP practices, including pair programming and TDD. The customer team used the defect-tracking system for the backlog, but most of the visibility of the stories was through index cards. Story cards were used during iteration planning meetings, and the task board tracked the progress.
Scrum 被用作组织和客户的外部报告机制。该团队进行了两周的迭代,大约每四个月发布一次产品。这取决于正在开发的功能。回顾是每次迭代计划会议的一部分,并对讨论的前三个优先事项采取行动。
Scrum was used as the outside reporting mechanism to the organization and the customers. The team had two week iterations and released the product about every four months. This varied depending on the functionality being developed. Retrospectives were held as part of every iteration planning session, and action was taken on the top three priority items discussed.
通过 CruiseControl 的持续集成为测试人员提供了持续的构建,并在每次迭代结束时进行了演示。每个测试人员都有一个用于测试 Web 应用程序的本地环境,但是系统可以使用三个测试环境。第一个是测试新故事,并根据需要使用最新版本进行了更新。第二个用于测试客户报告的问题,因为它已向客户发布了最后一个版本。第三个环境是一个完整的独立测试环境,可用于测试完整部署、通信链路以及固件和硬件。正是在这个环境中,我们运行了负载和可靠性测试。
Continuous integration through CruiseControl provided constant builds for the testers and the demonstrations held at the end of every iteration. Each tester had a local environment for testing the web application, but there were three test environments available to the system. The first one was to test new stories and was updated as needed with the latest build. The second one was for testing client-reported issues, because it had the last version released to the clients. The third environment was a full stand-alone test environment that was available for testing full deploys, communication links, and the firmware and hardware. It was on this environment that we ran our load and reliability tests.
驱动开发的测试包括单元测试和验收测试。
The tests driving development included unit test and acceptance tests.
单元测试是支持编程的面向技术的测试。那些作为测试驱动开发的一部分而开发的,不仅可以帮助程序员正确理解故事,还可以帮助设计系统。
Unit tests are technology-facing tests that support programming. Those that are developed as part of test-driven development not only help the programmer get the story right but also help to design the system.
第 7 章,“支持团队的面向技术的测试”,详细解释了单元测试和 TDD。
Chapter 7, “Technology-Facing Tests that Support the Team,” explains more about unit testing and TDD.
远程数据监控项目的程序员全心投入测试驱动开发 (TDD) 和结对编程。所有新功能都是使用结对编程开发和测试的。所有交付给测试人员的故事都得到了单元测试的支持,编码完成后发现的错误很少。发现的错误通常与集成相关。
The programmers on the Remote Data Monitoring project bought into Test Driven Development (TDD) and pair programming wholeheartedly. All new functionality was developed and tested using pair programming. All stories delivered to the testers were supported by unit tests, and very few bugs were found after coding was complete. The bugs that were found were generally integration-related.
然而,当团队刚开始时,遗留系统几乎没有单元测试来支持重构。随着流程变更的实施,开发人员决定开始解决这个问题。每次他们接触遗留系统中的一段代码时,他们都会添加单元测试并根据需要重构代码。渐渐地,遗留系统变得更加稳定,并且能够在需要时承受重大重构。我们体验了单元测试的力量!
However, when the team first started, the legacy system had few unit tests to support refactoring. As process changes were implemented, the developers decided to start fixing the problem. Every time they touched a piece of code in the legacy system, they added unit tests and refactored the code as necessary. Gradually, the legacy system became more stable and was able to withstand major refactoring when it was needed. We experienced the power of unit tests!
产品工程师(客户代理人)负责创建验收测试。这些测试的形式因实际情况而异。尽管一开始他很挣扎,但产品工程师非常擅长在程序员开始编码之前给他们进行测试。该团队创建了一个测试模板,它随着时间的推移而发展,同时满足了程序员和测试人员的需求。
The product engineer (the customer proxy) took ownership of creating the acceptance tests. These tests varied in format depending on the actual story. Although he struggled at first, the product engineer got pretty good at giving the tests to the programmers before they started coding. The team created a test template, which evolved over time, that met both the programmers’ and the testers’ needs.
测试有时是非正式编写的,但它们包括数据、需要的设置(如果不是很明显的话)、对故事至关重要的不同变体以及一些示例。该团队发现示例有助于阐明对许多故事的期望。
The tests were sometimes informally written, but they included data, required setup if it wasn’t immediately obvious, different variations that were critical to the story, and some examples. The team found that examples helped clarify the expectations for many of the stories.
测试团队尽快将验收测试自动化,通常是在开发故事的同时进行。当然,产品工程师可以回答开发过程中出现的任何问题。
The test team automated the acceptance tests as soon as possible, usually at the same time as the stories were being developed. Of course, the product engineer was available to answer any questions that came up during development.
这些验收测试有三个目的。它们是支持开发的面向业务的测试,因为它们是在编码开始之前提供给团队的。其次,测试团队将它们用作回归套件的自动化基础,并为探索性测试提供了未来的想法。第三个目的是确认实施满足了客户的需求。产品工程师做了这个方案验证。
These acceptance tests served three purposes. They were business-facing tests that supported development because they were given to the team before coding started. Secondly, they were used by the test team as the basis of automation that fed into the regression suite and provided future ideas for exploratory testing. The third purpose was to confirm that the implementation met the needs of the customer. The product engineer did this solution verification.
请参阅第 8 章,“支持团队的面向业务的测试”,了解有关使用验收测试推动开发的更多信息。
See Chapter 8, “Business-Facing Tests that Support the Team,” for more about driving development with acceptance tests.
自动化涉及功能测试结构、Web 服务和嵌入式测试。
Automation involved the functional test structure, web services, and embedded testing.
Ruby 与 Watir 一起用作功能自动化框架的首选工具。它被确定为被测系统所需的定制具有最大的灵活性和机会。
Ruby was used with Watir as the tool of choice for the functional automation framework. It was determined to have the most flexibility and opportunity for customization that was required for the system under test.
自动化测试代码包括三个不同的层,如图12-3所示。最低层,第 1 层,包括 Watir 和其他类,例如写入日志文件的记录器。
The automated test code included three distinct layers, shown in Figure 12-3. The lowest layer, Layer 1, included Watir and other classes, such as loggers that wrote to the log files.
第二层,第 2 层,是页面访问层,其中包含用于访问各个网页的代码的类。例如,在被测应用程序 (AUT) 中有一个登录页面、一个创建用户页面和一个编辑用户页面。用 Ruby 编写的类包含可以在 AUT 中执行某些功能的代码,例如登录应用程序的类、编辑用户的类以及为用户分配访问权限的类。这些类不包含任何数据。例如,登录类不知道用什么用户名登录。
The second layer, Layer 2, was the page access layer, where classes that contained code to access individual web pages lived. For example, in the application under test (AUT) there was a login page, a create user page, and an edit user page. Classes written in Ruby contained code that could perform certain functions in the AUT, such as a class that logs into the application, a class to edit a user, and a class to assign access rights to a user. These classes contained no data. For example, the log-in class didn’t know what username to log in with.
第三层也是顶层,第 3 层,是测试层,它包含执行测试所需的数据。它称为第 2 层类,后者又称为第 1 层。
The third and top layer, Layer 3, was the test layer, and it contained the data needed to perform a test. It called Layer 2 classes, which in turned called Layer 1.
例如,实际测试将调用 LogIn 并传递 Janet 作为用户名和 Passw0rd 作为密码。这意味着您可以轻松地输入许多不同的数据集。
For example, the actual test would call LogIn and pass Janet as the user name and Passw0rd as the password. This meant you could feed in many different data sets easily.
LogIn ('Janet', 'Passw0rd')
LogIn ('Janet', 'Passw0rd')
第 2 层还知道如何处理应用程序生成的错误消息。例如,当在登录页面上输入了无效的用户名时,登录类会检测到错误消息,然后将问题传递回第 3 层中的测试。
Layer 2 also knew how to handle the error messages the application generated. For example, when an invalid username was entered on the login page, the login class detected the error message and then passed the problem back to the tests in Layer 3.
这意味着相同的第 2 层类可用于快乐路径测试和负面测试。在否定的情况下,第 3 层会期望第 2 层返回失败,然后通过访问第 2 层从浏览器中抓取的错误消息来检查测试失败的原因是否正确。
This means the same Layer 2 classes could be used for both happy path testing and for negative testing. In the negative case, Layer 3 would expect Layer 2 to return a failure, and would then check to see if the test failed for the correct reason by accessing the error messages that Layer Two scraped from the browser.
功能测试使用 Ruby 和 Watir 来控制浏览器上的 DOM,并且可以访问页面中几乎所有的对象。自动化测试套件在每晚构建时运行,以向团队提供有关高级应用程序行为的一致反馈。当团队继续构建单元测试时,这是一个救星。该架构有效地适应了支持团队的面向业务的测试。
The functional tests used Ruby with Watir to control the DOM on the browser and could access almost all of the objects in the page. The automated test suite was run on nightly builds to give the team consistent feedback on high-level application behavior. This was a lifesaver as the team continued to build out the unit tests. This architecture efficiently accommodated the business-facing tests that support the team.
客户端使用 Web 服务来与他们的一些其他应用程序进行交互。开发组使用 Ruby 编写了一个客户端来测试他们开发的每个服务。对于这些测试,使用了 Ruby 的单元测试框架 Test::Unit。
Web services were used by clients to interface with some of their other applications. The development group used Ruby to write a client to test each service they developed. For these tests, Ruby’s unit testing framework, Test::Unit, was used.
测试团队扩展了 Web 服务测试以涵盖 1,000 多个不同的测试用例,并且仅需几分钟即可运行。他们在短时间内为团队提供了大量的报道。
The web services tests were expanded by the test team to cover more than 1,000 different test cases, and took just minutes to run. They gave the team an amazing amount of coverage in a short period of time.
该团队向客户展示了测试客户端,客户也决定使用它。然而,客户随后认为它对他们不起作用,所以他们开始编写自己的测试,尽管使用 Ruby 以一种更加临时的方式进行。
The team demonstrated the test client to the customers, who decided to use it as well. However, the customers subsequently decided it didn’t work for them, so they started writing their own tests, albeit in a much more ad hoc fashion using Ruby.
他们使用 IRB,Ruby 提供的交互式界面,并以探索性方法提供值。它为客户提供了一个交互式环境,用于发现哪些有效,哪些无效。它还让他们熟悉了 Ruby 以及我们的测试方式,这让他们对我们的测试更有信心。他们的大部分用户验收测试都是使用 IRB 完成的。
They used IRB, the interactive interface provided by Ruby, and fed values in an exploratory method. It gave the customer an interactive environment for discovering what worked and what didn’t. It also let them get familiar with Ruby and how we were testing, and it gave them much more confidence in our tests. Much of their User Acceptance Testing was done using IRB.
Web 服务测试的三种不同倾向服务于三个不同的目的。程序员用它来帮助测试他们的客户端并推动他们的开发。测试人员使用它以非常高效的自动化方式对产品进行评判方式,并且客户能够使用 IRB 测试交付给他们的 Web 服务。
Three different slants on the web services tests served three different purposes. The programmers used it to help test their client and drive their development. The testers used it to critique the product in a very efficient automated manner, and the customers were able to test the web services delivered to them using IRB.
除了 Web 界面外,RDM 系统还包含一个小型嵌入式设备,该设备使用各种协议与测量设备进行通信。使用 Ruby,开发了各种测试来测试其管理界面的一部分。该界面是类似于 FTP 的命令行系统。
In addition to the web interface, the RDM system consisted of a small embedded device that communicated with measuring equipment using various protocols. Using Ruby, various tests were developed to test part of its administrative interface. This interface was a command-line system similar to FTP.
这些数据驱动的测试包含在 Excel 电子表格中。Ruby 脚本将使用 OLE 接口从 Excel 中读取命令并将它们发送到嵌入式设备。然后脚本会将设备的响应与电子表格中保存的预期结果进行比较。错误以红色突出显示。这些自动化测试大约需要一个小时才能运行,而手动执行相同的测试则需要八个小时。
These data-driven tests were contained in an Excel spreadsheet. A Ruby script would read commands from Excel using the OLE interface and send them to the embedded device. The script would then compare the response from the device with the expected result, also held in the spreadsheet. Errors were highlighted in red. These automated tests took approximately one hour to run, while doing the same tests manually would take eight hours.
虽然这提供了很多测试范围,但实际上并没有测试使用设备的原因,即从 RTU 读取数据。模拟器是用 Ruby 编写的,带有 FOX (FXRuby) GUI。这允许将模拟数据馈送到设备中。由于模拟器可以远程控制,它被纳入自动化测试中,以锻炼嵌入式设备读取数据、响应错误条件以及在输入数据超过预定阈值时生成警报的能力。
While this provided a lot of test coverage, it didn’t actually test the reason the device was used, which was to read data from RTUs. A simulator was written in Ruby with a FOX (FXRuby) GUI. This allowed mock data to be fed into the device. Because the simulator could be controlled remotely, it was incorporated into automated tests that exercised the embedded device’s ability to read data, respond to error conditions, and generate alarms when the input data exceeded a predetermined threshold.
嵌入式测试技术性很强,但借助模拟器提供的强大功能,整个团队都能够参与到设备测试中。编写模拟器是为了支持测试团队的测试,但固件程序员发现它很有价值,并用它来帮助他的开发工作。这是一个积极的意外副作用。象限 2 支持团队的测试可能会结合各种技术,就像他们在这个项目中所做的那样。
Embedded testing is highly technical, but with the power provided by the simulator, the whole team was able to participate in testing the device. The simulator was written to support testing for the test team, but the programmer for the firmware found it valuable and used it to help with his development efforts as well. That was a positive unexpected side effect. Quadrant 2 tests that support the team may incorporate a variety of technologies, as they did in this project.
本节概述了评价产品的面向业务的测试。
The business-facing tests that critique the product are outlined in this section.
自动化测试简单易用,团队中的每个人都可以使用。可以运行单独的测试脚本来设置特定条件,允许有效无需花费大量时间手动输入数据即可完成探索性测试。这适用于所有三个测试框架:功能、Web 服务和嵌入式。
The automated tests were simple and easy for everyone on the team to use. Individual test scripts could be run to set up specific conditions, allowing effective exploratory testing to be done without having to spend a lot of time manually entering data. This worked for all three test frameworks: functional, web services, and embedded.
探索性测试、可用性测试和其他象限 3 测试在第 10 章“评价产品的面向业务的测试”中讨论。
Exploratory testing, usability testing, and other Quadrant 3 tests are discussed in Chapter 10, “Business-Facing Tests that Critique the Product.”
该团队进行了探索性测试以补充自动化测试套件并获得最佳覆盖率。这种与系统的人机交互发现了自动化没有发现的问题。
The team performed exploratory testing to supplement the automated test suites and get the best coverage possible. This human interaction with the system found issues that automation didn’t find.
可用性测试不是系统的关键要求,但测试人员会观察,以便界面有意义并且流畅。测试人员广泛使用探索性测试来评价产品。产品工程师还使用探索性测试来进行他的解决方案验证测试。
Usability testing was not a critical requirement for the system, but the testers watched so that the interface made sense and flowed smoothly. The testers used exploratory testing extensively to critique the product. The product engineer also used exploratory testing for his solution verification tests.
如图 12-2所示,来自系统的数据在 JMS 队列和 Web 浏览器上可用。为了测试 JMS 队列,开发组编写了一个 Java 代理。它连接到队列并将任何到达的数据打印到控制台。他们还编写了一个 Ruby 客户端,通过管道接收这些数据,然后在 Ruby 自动测试系统中可用。
As shown in Figure 12-2, the data from the system is available on a JMS queue, as well as the web browser. To test the JMS queue, the development group wrote a Java proxy. It connected to a queue and printed any arriving data to the console. They also wrote a Ruby client that received this data via a pipe, which was then available in the Ruby automated test system.
遇到警报情况时会自动发送电子邮件。警报电子邮件包含纯文本电子邮件和带附件的电子邮件。MIME附件包含对测试有用的数据,因此编写了支持附件的 Ruby 电子邮件客户端。
Emails were automatically sent when alarm conditions were encountered. The alarm emails contained both plain text email and email with attachments. The MIME attachments contained data useful for testing, so a Ruby email client that supported attachments was written.
象限 3 包括端到端功能测试,用于演示系统每个部分的预期行为。从一开始就很明显,只有使用所有组件才能确定整个远程数据监控系统是否正确运行。一旦编写了模拟器、嵌入式设备测试、Web 服务测试和应用程序测试,将它们组合起来生成整个系统的自动化测试就相对简单了。再次使用 Excel 电子表格来保存测试数据,并编写 Ruby 类来访问数据和预期结果。
Quadrant 3 includes end-to-end functional testing that demonstrates the desired behavior of every part of the system. From the beginning, it was apparent that correct operation of the whole Remote Data Monitoring system could only be determined when all components were used. Once the simulator, embedded device tests, web services tests, and application tests were written, it was a relatively simple matter to combine them to produce an automated test of the entire system. Once again, Excel spreadsheets were used to hold the test data, and Ruby classes were written to access the data and expected results.
卫星传输路径的不可预测响应使端到端测试变得复杂。设置了预定义的超时值,如果测试的实际值与预期值不匹配,测试将循环直到匹配或达到超时。当超时到期时,测试被视为失败。发现并消除了大多数传输问题这边走。手动测试不太可能发现它们,因为它们是零星的问题。
The end-to-end tests were complicated by the unpredictable response of the satellite transmission path. A predefined timeout value was set, and if the test’s actual value did not match the expected value, the test would cycle until it matched or the timeout was reached. When the timeout expired, the test was deemed to have failed. Most transmission issues were found and eliminated this way. It would have been highly unlikely that they would have been found with manual testing, because they were sporadic issues.
因为诸如此类的端到端测试可能很脆弱,所以它们可能不会作为自动回归套件的一部分保留。如果系统的所有组件都很好地覆盖了自动回归测试,则可能不需要自动端到端测试。但是,由于该系统的性质,如果没有自动化就不可能进行完整的测试。
Because end-to-end tests such as these can be fragile, they may not be kept as part of the automated regression suite. If all of the components of the system are well covered with automated regression tests, automated end-to-end tests might not be necessary. However, due to the nature of this system, it wasn’t possible to do a full test without automation.
用户验收测试 (UAT) 是客户对产品的最终评价,客户本应从一开始就参与到项目中。在此示例中,真正的客户在法国,距离开发团队数千英里。该团队必须具有创造力才能获得成功的 UAT。客户在这一年中与团队成员一起工作了几次,因此与他们从未见过面相比,他们能够更轻松地与团队互动。
User Acceptance Testing (UAT) is the final critique of the product by the customer, who should have been involved in the project from the start. In this example, the real customer was in France, thousands of miles from the development team. The team had to be inventive to have a successful UAT. The customer came to work with the team members a couple of times during the year and so was able to interact with the team a little easier than if they’d never met.
在团队引入敏捷开发后,Janet 前往法国,在客户现场促成第一个 UAT。它工作得相当好,在修复了一些关键问题后,该版本被接受了。团队从那次经历中学到了很多东西。
After the team introduced agile development, Janet went to France to facilitate the first UAT at the customer site. It worked fairly well, and the release was accepted after a few critical issues were fixed. The team learned a lot from that experience.
第二次 UAT 签核是在内部完成的。为了做好准备,该团队与客户合作开发了一组客户可以执行的测试来验证新功能。客户能够在整个开发周期中测试应用程序,因此 UAT 没有产生任何问题。客户来了,进行了测试,并在一天内签收了。
The second UAT sign-off was done in-house. To prepare, the team worked with the customer to develop a set of tests the customer could perform to verify new functionality. The customer was able to test the application throughout the development cycle, so UAT didn’t produce any issues. The customer came, ran through the tests, and signed off in a day.
我们怎么强调与客户合作的重要性都不为过。尽管产品工程师是客户的代理人,但与实际客户面对面交流至关重要。随着时间的推移建立的关系对于项目的成功至关重要。珍妮特坚信 UAT 的成功是因为客户知道团队一路走来所做的事情。
We cannot stress the importance of working with the customer enough. Even though the product engineer was the proxy for the customer, it was crucial to get face time with the actual customer. The relationship that had been built over time was critical to the success of the project. Janet strongly believes that the UAT succeeded because the customer knew what the team was doing along the way.
可靠性是第 4 象限测试解决的“问题”之一,它是系统的一个关键因素,因为它监视的是经常出现故障的远程站点人迹罕至,尤其是在冬天。为测试嵌入式系统而开发的模拟器被设置在一个单独的环境中,并且一次运行数周以测量整个系统的稳定性(又一个“ility”)。可以根据需要计划和编码对系统设计的更正。这是一个很好的例子,说明为什么您不应该等到项目结束才进行批评产品的面向技术的测试。
Reliability, one of the “ilities” addressed by Quadrant 4 tests, was a critical factor of the system because it was monitoring remote sites that were often inaccessible, especially in winter. The simulator that was developed for testing the embedded system was set up on a separate environment, and was run for weeks at a time measuring stability (yet another “ility”) of the whole system. Corrections to the system design could be planned and coded as needed. This is a good example of why you shouldn’t wait until the end of the project to do the technology-facing tests that critique the product.
请参阅第 10 章,“评估产品的面向业务的测试”,了解更多关于可靠性测试等象限 4 测试的信息。
See Chapter 10, “Business-Facing Tests that Critique the Product,” for more about Quadrant 4 tests such as reliability testing.
本节介绍了记录所采用的方法。
The approach taken to documentation is presented in this section.
在开发过程中,很明显测试代码需要一个正式的文档系统。最简单的解决方案是使用 RDoc,类似于 Javadoc,但用于 Ruby。RDoc 从源代码中提取带标签的注释,并生成包含文件、类和方法详细信息的网页。这些文档是每晚使用批处理文件生成的,并且可供整个团队使用。很容易找到创建了哪些测试装置。
During development, it became clear that a formal documentation system was needed for the test code. The simplest solution was to use RDoc, similar to Javadoc, but for Ruby. RDoc extracted tagged comments from the source code and generated web pages with details of files, classes, and methods. The documents were generated every night using a batch file and were available to the complete team. It was easy to find what test fixtures were created.
测试代码的文档有助于记录测试,并使我们更容易找到我们正在测试的内容以及测试做了什么。它非常强大且易于使用。
The documentation of the test code helped to document the tests and make it easier to find what we were testing and what the tests did. It was very powerful and easy to use.
尽管正在进行综合测试,但在测试团队之外几乎没有证据表明这一点。自动化测试期间生成的日志为跟踪问题提供了很好的信息,但不适合更广泛的受众。
Although comprehensive testing was being performed, there was little evidence of this outside of the test team. The logs generated during automated tests provided good information to track down problems but were not suitable for a wider audience.
第 16 章,“开始行动”,给出了团队报告测试结果的方式的更多示例。
Chapter 16, “Hit the Ground Running,” gives more examples of ways teams report test results.
为了提高正在执行的测试的可见性,测试团队使用 Apache、PHP 和 mySQL 开发了一个日志记录和报告系统。当测试运行时,它将结果记录到数据库中。Web 前端允许项目利益相关者确定运行了哪些测试、通过/失败率和其他信息。
To raise the visibility of the tests being performed, the test team developed a logging and reporting system using Apache, PHP, and mySQL. When a test ran, it logged the result into the database. A web front end allowed project stakeholders to determine what tests were run, the pass/failure rate, and other information.
Chapter 18, “Coding and Testing,” also discusses uses of big visible charts.
我们还相信尽可能让我们的进步(好或坏)可见。为此,我们在此过程中创建了图表和图形,并将它们张贴在公共区域。图 12-4显示了我们创建的一些图表。
We also believed in making our progress visible (good or bad) as much as possible. To this end we created charts and graphs along the way and posted them in common areas. Figure 12-4 shows some of the charts we created.
此示例演示了如何在复杂开发项目的生命周期中结合来自所有四个敏捷测试象限的测试实践以实现成功交付。这个团队的经验说明了我们一直强调的许多原则。整个团队,包括程序员、测试人员、客户代理人和实际客户,都为解决自动化问题做出了贡献。他们尝试了不同的方法。他们以不同的方式结合了自己开发的和开源的工具来执行所有级别的测试,从单元级别到端到端系统测试和 UAT。项目的成功证明了测试方法的成功。
This example demonstrates how testing practices from all four agile testing quadrants are combined during the life of a complex development project to achieve successful delivery. The experience of this team illustrates many of the principles we have been emphasizing. The whole team, including programmers, testers, customer proxy, and the actual customer, contributed to efforts to solve automation problems. They experimented with different approaches. They combined their homegrown and open source tools in different ways to perform testing at all levels, from the unit level to end-to-end system testing and UAT. The success of the project demonstrates the success of the testing approach.
当您计划每个史诗、发布或迭代时,与您的客户团队合作以了解业务优先级并分析风险。使用象限来帮助确定将需要的所有不同类型的测试以及何时应该执行它们。性能是最重要的标准吗?是个最高优先级与其他系统接口的能力?可用性可能是最重要的方面吗?
As you plan each epic, release, or iteration, work with your customer team to understand the business priorities and analyze risks. Use the quadrants to help identify all of the different types of testing that will be needed and when they should be performed. Is performance the most important criteria? Is the highest priority the ability to interface with other systems? Is usability perhaps the most important aspect?
投资于适应被测系统复杂性的测试架构。计划在适当的时间获得必要的资源和专业知识以进行专门测试。对于每种类型的测试,您的团队应该共同努力选择能够解决您的测试问题的工具。使用回顾来持续评估您的团队是否拥有成功所需的资源,以及是否及时指定了所有必要的测试以实现其目的,并适当地自动化。
Invest in a test architecture that accommodates the complexity of the system under test. Plan to obtain necessary resources and expertise at the right time for specialized tests. For each type of test, your team should work together to choose tools that solve your testing problems. Use retrospectives to continually evaluate whether your team has the resources it needs to succeed, and whether all necessary tests are being specified in time to serve their purpose, and automated appropriately.
端到端测试似乎不可能做到吗?您的团队是否发现很难编写单元测试?正如 Janet 的团队所做的那样,让每个人都尝试不同的方法和工具。这些象限提供了一个框架,可以就创造性方法进行富有成效的头脑风暴,以实现让团队为业务创造价值的测试。
Does end-to-end testing seem impossible to do? Is your team finding it hard to write unit tests? As Janet’s team did, get everyone experimenting with different approaches and tools. The quadrants provide a framework for productive brainstorming on creative ways to achieve the testing that will let the team deliver value to the business.
在本章中,我们描述了一个真实的项目,该项目使用来自所有四个敏捷测试象限的测试来克服困难的测试挑战。我们使用该项目中的示例来展示团队如何成功进行所有类型的测试。远程数据监控系统项目的一些重要经验教训是:
In this chapter, we described a real project that used tests from all four agile testing quadrants to overcome difficult testing challenges. We used examples from this project to show how teams can succeed with all types of testing. Some important lessons from the Remote Data Monitoring System project are:
整个团队应该选择或创建解决每个测试问题的工具。
The whole team should choose or create tools that solve each testing problem.
可能需要结合使用电子表格和自定义编写的测试脚本等常用业务工具来完成复杂的测试。
Combinations of common business tools such as spreadsheets and custom-written test scripts may be needed to accomplish complex tests.
花时间构建适用于所有团队成员的正确测试架构。
Invest time in building the right test architecture that works for all team members.
想方设法让客户参与所有类型的测试,即使他们身处偏远地区。
Find ways to keep customers involved in all types of testing, even if they’re in a remote location.
以一种让所有涉众都了解迭代和项目进度的方式报告测试结果。
Report test results in a way that keeps all stakeholders informed about the iteration and project progress.
不要忘记记录 . . . 但只有有用的东西。
Don’t forget to document . . . but only what is useful.
考虑整个开发周期中测试的所有四个象限。
Think about all four quadrants of testing throughout your development cycles.
使用在测试期间吸取的经验教训来评论产品,以推动后续迭代中的开发。
Use lessons learned during testing to critique the product in order to drive development in subsequent iterations.
测试自动化是核心的敏捷实践。敏捷项目依赖于自动化。足够好的自动化可以让团队腾出时间来频繁地交付高质量的代码。它提供了一个框架,让团队在保持高标准的同时最大限度地提高速度。源代码控制、自动化构建和测试套件、部署、监控以及各种脚本和工具消除了单调乏味,确保了可靠性,并让团队始终尽最大努力工作。
Test automation is a core agile practice. Agile projects depend on automation. Good-enough automation frees the team to deliver high-quality code frequently. It provides a framework that lets the team maximize its velocity while maintaining a high standard. Source code control, automated builds and test suites, deployment, monitoring, and a variety of scripts and tools eliminate tedium, ensure reliability, and allow the team to do its best work at all times.
自动化也是一个广泛的话题。它包括编写简单的 shell 脚本、设置会话属性和创建强大的自动化测试等任务。随着我们了解更好的软件开发方法,自动化工具的范围和数量似乎呈指数级增长。令人高兴的是,教授自动化方法的优秀书籍的数量似乎增长得同样快。
Automation is also a vast topic. It includes tasks like writing simple shell scripts, setting up session properties, and creating robust automated tests. The range and number of automated tools seem to grow exponentially as we learn about better ways to produce software. Happily, the number of excellent books that teach ways to automate appears to grow just as fast.
本书着重于测试人员在敏捷开发中的角色。因为自动化是敏捷开发成功的关键,所以我们需要讨论它,但我们不能开始涵盖该主题的每个方面。我们确实想解释的是为什么您作为测试人员必须拥抱自动化,以及您和您的团队如何克服可能阻碍您的自动化工作的许多障碍。本节介绍如何应用敏捷价值观、原则和实践来发展实用的自动化策略、克服障碍并推动测试自动化。
This book is focused on the tester’s role in agile development. Because automation is key to successful agile development, we need to talk about it, but we can’t begin to cover every aspect of the subject. What we do want to explain is why you, as a tester, must embrace automation, and how you and your team can overcome the many obstacles that can hamper your automation efforts. This section describes how you can apply agile values, principles, and practices to grow a practical automation strategy, overcome barriers, and get traction on test automation.
为什么我们要自动化测试、构建过程、部署和其他任务?敏捷团队专注于始终拥有可工作的软件,这使他们能够根据需要经常发布生产就绪的软件。实现这个目标需要不断的测试。在本章中,我们将探讨我们想要实现自动化的原因以及难以实现自动化的挑战。
Why do we automate testing, the build process, deployment, and other tasks? Agile teams focus on always having working software, which enables them to release production-ready software as often as needed. Achieving this goal requires constant testing. In this chapter, we look at reasons we want to automate and the challenges that make it hard to get traction on automation.
除了我们所说的您需要自动化才能成功使用敏捷之外,还有多种原因可以实现自动化。我们的列表包括以下内容:
There are multiple reasons to automate besides our saying you need to have automation to be successful using agile. Our list includes the following:
Manual testing takes too long.
Manual processes are error prone.
Automation frees people to do their best work.
Automated regression tests provide a safety net.
Automated tests give feedback early and often.
Tests and examples that drive coding can do more.
自动化可以带来良好的投资回报。
Automation can be a good return on investment.
让我们更详细地探讨其中的每一个。
Let’s explore each of these in a little more detail.
团队想要自动化的最基本原因是手动完成所有必要的测试需要太长时间。随着您的应用程序变得越来越大,测试所有内容的时间也越来越长,有时呈指数增长,具体取决于 AUT(被测应用程序)的复杂性。
The most basic reason a team wants to automate is that it simply takes too long to complete all of the necessary testing manually. As your application gets bigger and bigger, the time to test everything grows longer and longer, sometimes exponentially, depending on the complexity of the AUT (Application under test).
敏捷团队能够通过每天拥有生产就绪的软件,在每次短迭代结束时交付生产就绪的软件。至少每天运行一整套通过回归测试是必不可少的做法,而手动回归测试无法做到这一点。如果您现在没有任何自动化,您将不得不手动进行回归测试,但不要让它阻止您开始自动化。
Agile teams are able to deliver production-ready software at the end of each short iteration by having production-ready software every day. Running a full suite of passing regression tests at least daily is an indispensable practice, and you can’t do it with manual regression testing. If you don’t have any automation now, you’ll have to regression test manually, but don’t let that stop you from starting to automate it.
如果您手动执行回归测试,那么每天、每次迭代都会花费越来越多的时间进行测试。为了让测试跟上编码的步伐,要么程序员必须花时间帮助进行手动回归测试,要么团队必须雇用更多的测试人员。不可避免地,技术债务和挫败感都会增加。
If you execute your regression testing manually, it takes more and more time testing every day, every iteration. In order for testing to keep pace with coding, either the programmers have to take time to help with manual regression testing, or the team has to hire more testers. Inevitably, both technical debt and frustration will grow.
如果代码甚至不必通过一套自动化的单元级回归测试,测试人员可能会花费大量时间研究、尝试重现和报告那些简单的错误,而花更少的时间来发现潜在的严重系统级错误。此外,由于团队没有进行测试先行的开发,代码设计的可测试性更差,可能无法提供业务所需的功能。
If the code doesn’t even have to pass a suite of automated unit level regression tests, the testers will probably spend much of their time researching, trying to reproduce and report those simple bugs, and less time finding potentially serious system level bugs. In addition, because the team isn’t doing test-first development, code design is more likely to be less testable and may not provide the functionality desired by the business.
手动测试许多不同的场景可能会花费很多时间,尤其是当您将输入键入用户界面时。如果您没有自动化的方法来加速,那么为各种复杂场景设置数据可能是一项艰巨的任务。结果,可能只测试了有限数量的场景,并且可能会遗漏重要的缺陷。
Manually testing a number of different scenarios can take a lot of time, especially if you’re keying inputs into a user interface. Setting up data for a variety of complex scenarios can be an overwhelming task if you have no automated way to speed it up. As a result, only a limited number of scenarios may be tested, and important defects can be missed.
手动测试变得重复,特别是如果您遵循脚本测试,并且手动测试很快就会变得无聊。犯错太容易了,即使是简单的错误也会被忽视。将跳过步骤甚至整个测试。如果团队面临紧迫的截止日期,就会有偷工减料的诱惑,结果就是错过了问题。
Manual testing gets repetitive, especially if you’re following scripted tests, and manual tests get boring very quickly. It’s way too easy to make mistakes and overlook even simple bugs. Steps and even entire tests will be skipped. If the team’s facing a tight deadline, there’s a temptation to cut corners, and the result is a missed problem.
由于手动测试很慢,您可能仍在迭代的最后一天的午夜进行测试。那么你会注意到多少错误?
Because manual testing is slow, you might still be testing at midnight on the last day of the iteration. How many bugs will you notice then?
自动化构建、部署、版本控制和监控也大大有助于降低风险并使您的开发过程更加一致。自动化这些脚本化测试消除了出错的可能性,因为每次测试都是以完全相同的方式完成的。
Automated builds, deployment, version control, and monitoring also go a long way toward mitigating risk and making your development process more consistent. Automating these scripted tests eliminate the possibility of errors, because each test is done exactly the same way every time.
“一次构建,多次部署”的格言是测试人员的梦想成真。构建和部署过程的自动化使您能够准确了解您在任何给定环境中测试的内容。
The adage of “build once, deploy to many” is a tester’s dream come true. Automation of the build and deploy processes allow you to know exactly what you are testing on any given environment.
编写测试先行的代码有助于程序员理解需求并相应地设计代码。让持续构建运行所有单元测试和功能回归测试意味着有更多时间进行有趣的探索性测试。自动化探索性测试的设置意味着有更多时间来探索系统的潜在薄弱部分。因为您没有花时间执行繁琐的手动脚本,所以您有精力做好工作,思考不同的场景并更多地了解应用程序的工作原理。
Writing code test-first helps programmers understand requirements and design code accordingly. Having continual builds run all of the unit tests and the functional regression tests means more time to do interesting exploratory testing. Automating the setup for exploratory tests means even more time to probe into potentially weak parts of the system. Because you didn’t spend time executing tedious manual scripts, you have the energy to do a good job, thinking of different scenarios and learning more about how the application works.
如果我们一直在思考如何对修复或新功能进行自动化测试,我们更有可能想到可测试性和质量设计,而不是可能证明脆弱的快速破解。这意味着更好的代码和更好的测试。
If we’re thinking constantly about how to automate tests for a fix or new feature, we’re more likely to think of testability and a quality design rather than a quick hack that might prove fragile. That means better code and better tests.
自动化测试实际上可以帮助实现整个应用程序的一致性。
Automating tests can actually help with consistency across the application.
有关 Ruby 和 Watir 的更多信息,请参阅第 9 章“支持团队的面向业务的测试工具包”、第 12 章“测试象限总结”和第 14 章“敏捷测试自动化策略”。
See Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” Chapter 12, “Summary of Testing Quadrants,” and Chapter 14, “An Agile Test Automation Strategy,” for more information about Ruby and Watir.
Pragmatic Project Automation [2004]等书籍可以指导您自动执行日常开发工作,让您的团队腾出时间进行重要活动,例如探索性测试。
Books such as Pragmatic Project Automation [2004] can guide you in automating daily development chores and free your team for important activities such as exploratory testing.
当好人可以自由地做最好的工作时,项目就会成功。适当地自动化测试可以实现这一点。检测现有功能变化并提供即时反馈的自动回归测试是其中的主要组成部分。
Projects succeed when good people are free to do their best work. Automating tests appropriately makes that happen. Automated regression tests that detect changes to existing functionality and provide immediate feedback are a primary component of this.
大多数在软件行业工作了几年的从业者都知道,当他们面临修复错误或在设计不佳的代码中实现自动化测试未涵盖的新功能时的恐惧感。在这里挤压气球的一端,另一部分就会凸出。会坏吗?
Most practitioners who’ve been in the software business for a few years know the feeling of dread when they’re faced with fixing a bug or implementing a new feature in poorly designed code that isn’t covered by automated tests. Squeeze one end of the balloon here and another part of it bulges out. Will it break?
通过自动回归测试知道代码具有足够的覆盖率,给人以极大的信心。当然,更改可能会产生意想不到的效果,但如果是在单元级别,我们会在几分钟内知道,如果是在更高的功能级别,我们会在几小时内知道。让更改测试先行意味着在编写代码之前仔细考虑更改的行为并编写测试来验证它,这增加了这种信心。
Knowing the code has sufficient coverage by automated regression tests gives a great feeling of confidence. Sure, a change might produce an unexpected effect, but we’ll know about it within a matter of minutes if it’s at the unit level, or hours if at a higher functional level. Making the change test-first means thinking through the changed behavior before writing the code and writing a test to verify it, which adds to that confidence.
当他们没有作为安全网的自动化测试套件时,程序员可能会开始将测试人员本身视为安全网。很容易想象 Joe Programmer 的思考过程是这样的:“我应该返回并为 formatEmployeeInfo 添加一些自动化单元测试,但我知道 Susie Tester 将检查每个手动使用它的页面。她会看看是否有任何问题,所以我只是在重复她的努力。”
When they don’t have an automated suite of tests acting as a safety net, the programmers may start viewing the testers themselves as a safety net. It’s easy to imagine that Joe Programmer’s thought process goes like this: “I ought to go back and add some automated unit tests for formatEmployeeInfo, but I know Susie Tester is going to check every page where it’s used manually. She’ll see if anything is off, so I’d just be duplicating her effort.”
程序员对测试人员的才能评价如此之高是件好事,但 Joe 正在走下坡路。如果他不自动化这些单元测试,他可能会跳过哪些其他测试?Susie 将非常忙于浏览所有这些页面。
It’s nice that a programmer would think so highly of the tester’s talents, but Joe is headed down a slippery slope. If he doesn’t automate these unit tests, which other tests might he skip? Susie is going to be awfully busy eyeballing all those pages.
自动化回归测试覆盖率高的团队可以无所畏惧地更改代码。他们不必怀疑,“如果我更改这个 formatEmployeeInfo 模块,我会破坏用户界面中的某些东西吗?” 测试会立即告诉他们是否损坏了任何东西。他们可以比完全依赖手动测试的团队快得多。
Teams that have good coverage from automated regression tests can make changes to the code fearlessly. They don’t have to wonder, “If I change this formatEmployeeInfo module, will I break something in the user interface?” The tests will tell them right away whether or not they broke anything. They can go lots faster than teams relying exclusively on manual testing.
一项功能的自动化测试通过后,它必须继续通过,直到功能被有意更改。当我们计划对应用程序进行更改时,我们会更改测试以适应它们。当自动化测试意外失败时,代码更改可能会引入回归缺陷。每次签入新代码时运行一套自动化测试有助于确保快速发现回归错误。快速反馈意味着更改在一些程序员的脑海中仍然是新鲜的,因此与直到几周后的某个测试阶段才发现错误相比,故障排除会更快。快速失败意味着修复错误的成本更低。
After an automated test for a piece of functionality passes, it must continue to pass until the functionality is intentionally changed. When we plan changes in the application, we change the tests to accommodate them. When an automated test fails unexpectedly, a regression defect may have been introduced by a code change. Running an automated suite of tests every time new code is checked in helps ensure that regression bugs will be caught quickly. Quick feedback means the change is still fresh in some programmer’s mind, so troubleshooting will go more quickly than if the bug weren’t found until some testing phase weeks later. Failing fast means bugs are cheaper to fix.
自动化测试定期运行,通常充当您的变更检测器。它们让团队有机会了解自上次构建以来发生了什么变化。例如,上一次构建是否有任何负面影响?如果您的自动化套件具有足够的覆盖范围,它可以很容易地分辨出手动测试人员永远无法找到的深远影响。
Automated tests run regularly and often act as your change detector. They allow the team an opportunity to know what has changed since the last build. For example, were there any negative side effects with the last build? If your automation suite has sufficient coverage, it can easily tell far-reaching effects that manual testers can never hope to find.
通常情况下,如果回归测试不是自动化的,它们就不会在每次迭代中运行,更不用说每天运行了。问题在比赛结束时很快出现,此时团队需要完成所有回归测试。本应及早发现的错误在游戏后期才被发现。早期测试的许多好处都失去了。
More often than not, if regression tests are not automated, they won’t get run every iteration, let alone every day. The problem arises very quickly during the end game, when the team needs to complete all of the regression tests. Bugs that would have been caught early are found late in the game. Many of the benefits of testing early are lost.
在第 7 章“支持团队的面向技术的测试”中,我们讨论了使用测试和示例来驱动编码。我们已经讨论了通过单元测试和客户测试来驱动编码的重要性。我们还想强调,如果这些测试是自动化的,它们会因为不同的原因而变得有价值。它们成为非常强大的回归套件的基础。
In Chapter 7, “Technology-Facing Tests that Support the Team,” we talked about using tests and examples to drive coding. We’ve talked about how important it is to drive coding with both unit and customer tests. We also want to stress that if these tests are automated, they become valuable for a different reason. They become the base for a very strong regression suite.
TDD 和 SDD(故事测试驱动开发)让团队始终以测试为先。在计划会议期间,他们会讨论测试和进行测试的最佳方法。他们设计代码使测试通过,因此可测试性从来都不是问题。自动化测试套件随着代码库一起增长,为持续重构提供安全网。整个团队实践 TDD 并始终如一地编写单元测试很重要,否则安全网就会出现漏洞。
TDD and SDD (story test-driven development) keep teams thinking test-first. During planning meetings, they talk about the tests and the best way to do them. They design code to make the tests pass, so testability is never an issue. The automated test suite grows along with the code base, providing a safety net for constant refactoring. It’s important that the whole team practices TDD and consistently writes unit tests, or the safety net will have holes.
参考书目包含 Jennitta Andrea [2008] 的一篇关于 TDD 团队礼仪的文章。
The bibliography contains an article by Jennitta Andrea [2008] on team etiquette for TDD.
团队也不会积累太多技术债务,而且他们的速度必然会稳定甚至随着时间的推移而增加。这就是为什么业务经理应该乐于让软件团队花时间正确实施良好实践的原因之一。
The team also doesn’t accrue too much technical debt, and their velocity is bound to be stable or even increase over time. That’s one of the reasons why the business managers should be happy to let software teams take the time to implement good practices correctly.
在第 III 部分中,我们解释了敏捷团队如何使用示例和测试来指导开发。当说明所需行为示例的测试自动化时,它们将成为系统实际工作方式的“活”文档。最好有关于某项功能如何工作的叙述性文档,但没有人会反对用红色和绿色显示代码如何在给定输入集上运行的可执行测试。
In Part III, we explained how agile teams use examples and tests to guide development. When tests that illustrate examples of desired behavior are automated, they become “living” documentation of how the system actually works. It’s good to have narrative documentation about how a piece of functionality works, but nobody can argue with an executable test that shows in red and green how the code operates on a given set of inputs.
静态文档很难保持最新,但如果我们在系统发生变化时不更新我们的自动化测试,测试就会失败。我们需要修复它们以保持我们的构建过程“绿色”。这意味着自动化测试始终是我们代码工作方式的准确描述。这只是我们在自动化方面的投资获得回报的方式之一。
It’s hard to keep static documentation up to date, but if we don’t update our automated tests when the system changes, the tests fail. We need to fix them to keep our build process “green.” This means that automated tests are always an accurate picture of how our code works. That’s just one of the ways our investment in automation pays off.
刚才提出的所有原因都有助于自动化的底线和回报。自动化为项目提供了一致性,并为团队提供了以不同方式进行测试并突破应用程序极限的机会。自动化意味着测试人员和团队成员有更多时间专注于及时将正确的产品推向市场。
All of the reasons just presented contribute to the bottom line and the payback of automation. Automation provides consistency to a project and gives the team opportunity to test differently and push the limits of the application. Automation means extra time for testers and team members to concentrate on getting the right product out to market in a timely manner.
测试自动化回报的一个重要组成部分是修复缺陷的方式。依赖手动测试的团队往往在编写包含错误的代码很久之后才发现错误。他们进入修复“当天的错误”的模式,而不是查看错误的根本原因并相应地重新设计代码。当程序员在他们自己的沙盒中运行自动化测试套件时,自动化回归测试会在代码签入之前发现错误,因此有时间纠正设计。这是一个更大的回报,也是你减少技术债务和开发可靠代码的方式。
An important component of test automation payback is the way defects are fixed. Teams that rely on manual tests tend to find bugs long after the code containing the bug is written. They get into the mode of fixing the “bug of the day,” instead of looking at the root cause of the bug and redesigning the code accordingly. When programmers run the automated test suite in their own sandbox, the automated regression tests find the bugs before the code is checked in, so there’s time to correct the design. That’s a much bigger payback, and it’s how you reduce technical debt and develop solid code.
早在 2001 年,Bret Pettichord [ 2001 ] 就列出了困扰自动化的七个问题。它们仍然适用,但适用于未将自动化作为其开发一部分的团队。当然,因为你在做敏捷,所以你在做,对吧?
Back in 2001, Bret Pettichord [2001] listed seven problems that plague automation. They are still applicable, but are intended for teams that do not incorporate automation as part of their development. And of course, because you are doing agile, you are doing that, right?
我们认为每个人都将自动化任务作为每个故事的一部分,但事实是,如果一切都在您的控制之下,您可能不会阅读本节。我们已经包含了 Bret 的列表,以显示如果您不将自动化作为日常项目可交付成果的一部分,您可能遇到的问题。
We would like to think that everyone has included automation tasks as part of each story, but the reality is that you probably wouldn’t be reading this section if you had it all under control. We’ve included Bret’s list to show what problems you probably have if you don’t include automation as part of the everyday project deliverables.
Bret 的自动化问题列表如下所示:
Bret’s list of automation problems looks like this:
仅将业余时间用于测试自动化并不能为其提供所需的重点。
Only using spare time for test automation doesn’t give it the focus it needs.
缺乏明确的目标。
There is a lack of clear goals.
缺乏经验。
There is a lack of experience.
人员流动率很高,因为你失去了你可能拥有的任何经验。
There is high turnover, because you lose any experience you may have.
对绝望的反应通常是选择自动化的原因,在这种情况下,它可能更像是一个愿望,而不是一个现实的建议。
A reaction to desperation is often the reason why automation is chosen, in which case it can be more of a wish than a realistic proposal.
There can be a reluctance to think about testing; the fun is in the automating, not in the testing.
专注于解决技术问题可能会使您忽视结果是否满足测试需要。
Focusing on solving the technology problem can cause you to lose sight of whether the result meets the testing need.
我们认为团队在尝试自动化时还会遇到其他一些问题。即使我们确实尝试在我们的项目可交付成果中包含自动化,也存在其他阻碍成功的障碍。在下一节中,我们列出了成功测试自动化的障碍。
We think there are some other problems that teams run into when trying to automate. Even if we do try to include automation in our project deliverables, there are other barriers to success. In the next section, we present our list of obstacles to successful test automation.
我们列出的成功测试自动化的障碍是基于我们自己的敏捷团队以及我们认识的其他团队的经验。
Our list of barriers to successful test automation is based on the experiences we’ve had with our own agile teams as well as that of the other teams we know.
习惯于在传统环境中工作的程序员,在传统环境中,一些独立的、看不见的 QA 团队进行所有测试,甚至可能不会对功能测试自动化进行太多思考。有些程序员懒得做太多测试,因为他们有 QA 团队作为安全网,可以在发布前发现错误。长的瀑布式开发周期使得测试对程序员来说更加遥远。当看不见的测试人员开始工作时,程序员已经转移到下一个版本。缺陷排在队列中,以后要花费巨大的代价来修复,而且没有人对产生它们负责。即便是采用了测试驱动开发,习惯于在单元级别进行自动化测试的程序员,也可能不会去思考超出单元级别的验收测试是如何完成的。
Programmers who are used to working in a traditional environment, where some separate, unseen QA team does all of the testing, may not even give functional test automation a lot of thought. Some programmers don’t bother to test much because they have the QA team as a safety net to catch bugs before release. Long waterfall development cycles make testing even more remote to programmers. By the time the unseen testers are doing their job, the programmers have moved on to the next release. Defects go into a queue to be fixed later at great expense, and nobody is accountable for having produced them. Even programmers who have adopted test-driven development and are used to automating tests at the unit level may not think about how acceptance tests beyond the unit level get done.
教育是让程序员和团队其他成员了解自动化重要性的关键。
Education is the key to getting programmers and the rest of the team to understand the importance of automation.
学习测试自动化是很困难的,尤其是学习如何以一种对投入其中的资源产生良好回报的方式进行测试。我们听 Brian Marick 用来描述开发人员(包括测试人员)必须克服的自动化初始阶段的一个术语是“痛苦的驼峰”(见图 13-1 )。这个短语指的是大多数团队在采用自动化时所经历的挣扎。
It’s hard to learn test automation, especially to learn how to do it in a way that produces a good return on the resources invested in it. A term we’ve heard Brian Marick use to describe the initial phase of automation that developers (including testers) have to overcome is the “hump of pain” (see Figure 13-1). This phrase refers to the struggle that most teams go through when adopting automation.
通常期望新团队采用 TDD 和重构等难以学习的实践。如果没有良好的指导、充足的时间来掌握新技能以及强大的管理支持,他们很容易灰心丧气。如果他们在学习上有额外的障碍,例如必须使用设计不佳的遗留代码,那么似乎不可能在测试自动化上获得牵引力。
New teams are often expected to adopt practices such as TDD and refactoring, which are difficult to learn. Without good coaching, plenty of time to master new skills, and strong management support, they’re easily discouraged. If they have extra obstacles to learning, such as having to work with poorly designed legacy code, it may seem impossible to ever get traction on test automation.
由于您正在构建特定领域的测试框架或学习新的功能测试工具,因此可能会出现痛苦的驼峰。您可能需要聘请专家来帮助您正确设置。
The hump of pain may occur because you are building your domain-specific testing framework or learning your new functional test tool. You may want to bring in an expert to help you get it set up right.
你知道你的团队已经克服了当自动化变得(如果不容易)至少是一个自然而根深蒂固的过程时的“驼峰”。Lisa 曾在三个成功采用 TDD 和功能测试自动化的团队工作过。每一次,团队都需要大量的时间、培训、承诺和鼓励来推动实践。
You know your team has overcome the “hump” when automation becomes, if not easy, at least a natural and ingrained process. Lisa has worked on three teams that successfully adopted TDD and functional test automation. Each time, the team needed lots of time, training, commitment, and encouragement to get traction on the practices.
即使整个团队都在努力解决这个问题,自动化也需要大量投资,而且可能不会立即得到回报。决定使用什么测试框架以及是在内部构建它们还是使用外部生产的工具需要时间和研究。可能需要新的硬件和软件。团队成员可能需要一段时间才能熟悉如何使用自动化测试工具。
Even with the whole team working on the problem, automation requires a big investment, one that may not pay off right away. It takes time and research to decide on what test frameworks to use and whether to build them in-house or use externally produced tools. New hardware and software are probably required. Team members may take a while to ramp up on how to use automated test harnesses.
许多人都经历过没有回报的测试自动化努力。他们的组织可能已经购买了供应商捕获回放工具,并将其提供给 QA 团队,并期望它能够解决所有自动化问题。这些工具通常放在架子上积尘。可能已经生成了数千行 GUI 测试脚本,没有人知道它们在做什么,或者无法维护的测试脚本不再有用。
Many people have experienced test automation efforts that didn’t pay off. Their organization may have purchased a vendor capture-playback tool, given it to the QA team, and expected it to solve all of the automation problems. Such tools often sit on a shelf gathering dust. There may have been thousands of lines of GUI test scripts generated, with no one left who knows what they do, or the test scripts that are impossible to maintain are no longer useful.
测试设计技能对自动化是否立即获得回报有着巨大的影响。糟糕的做法会产生难以理解和维护的测试,并可能产生难以解释的结果或需要时间研究的错误失败。培训和技能不足的团队可能会认为他们的自动化投资回报不值得他们花时间。
Test design skills have a huge impact on whether automation pays off right away. Poor practices produce tests that are hard to understand and maintain, and may produce hard-to-interpret results or false failures that take time to research. Teams with inadequate training and skills might decide the return on their automation investment isn’t worth their time.
良好的测试设计实践产生简单、设计良好、不断重构、可维护的测试。测试模块和对象的库会随着时间的推移而建立起来,并使新测试的自动化速度更快。有关自动化测试设计的一些提示和指南,请参阅第 14 章。
Good test design practices produce simple, well-designed, continually refactored, maintainable tests. Libraries of test modules and objects build up over time and make automating new tests quicker. See Chapter 14 for some hints on and guidelines for test design for automation.
我们知道捕获指标并不容易。例如,试图捕获编写和维护自动化测试所花费的时间与手动运行相同回归测试所花费的时间几乎是不可能的。同样,试图捕捉引入缺陷后几分钟内修复缺陷的成本与迭代结束后发现和修复问题的成本也相当困难。许多团队不努力跟踪这些信息。如果没有数字表明自动化需要更少的努力并提供更多的价值,团队就更难说服管理层相信对自动化的投资是值得的。缺乏证明自动化投资回报的指标也使得改变团队的习惯变得更加困难。
We know it’s not easy to capture metrics. For example, trying to capture the time it takes to write and maintain automated tests versus the time it takes to run the same regression tests manually is almost impossible. Similarly, trying to capture how much it costs to fix defects within minutes of introducing them versus how much it costs to find and fix problems after the end of the iteration is also quite difficult. Many teams don’t make the effort to track this information. Without numbers showing that automating requires less effort and provides more value, it’s harder for teams to convince management that an investment in automation is worthwhile. A lack of metrics that demonstrate automation’s return on investment also makes it harder to change a team’s habits.
通过用户界面进行自动化测试很棘手,因为 UI 在开发过程中往往会频繁更改。这就是简单的记录和回放技术很少成为敏捷项目好的选择的原因之一。
Automating tests through the user interface is tricky, because UIs tend to change frequently during development. That’s one reason that simple record and playback techniques are rarely a good choice for an agile project.
如果团队努力在底层业务逻辑和数据库访问方面做出良好的设计,并且经常进行重大返工,那么即使在 API 级别的 GUI 后面进行自动化测试也可能很难跟上。如果在设计系统时很少考虑测试,那么找到一种自动化测试的方法可能会很困难而且成本很高。程序员和测试人员需要一起工作以获得可测试的应用程序。
If the team is struggling to produce a good design on the underlying business logic and database access, and major rework is done frequently, it might be hard to keep up even with tests automated behind the GUI at the API level. If little thought is given to testing while designing the system, it might be difficult and expensive to find a way to automate tests. The programmers and testers need to work together to get a testable application.
尽管实际的代码和实现,如 GUI,在敏捷开发中往往会经常变化,但代码的意图很少改变。根据应用程序的意图而不是其实现来组织测试代码,使您能够跟上开发的步伐。
Although the actual code and implementation, like the GUI, tends to change frequently in agile development, the intent of code rarely changes. Organizing test code by the application’s intent, rather than by its implementation, allows you to keep up with development.
在第 14 章“敏捷测试自动化策略”中,我们将研究组织自动化测试的方法。
In Chapter 14, “An Agile Test Automation Strategy,” we’ll look at ways to organize automated tests.
根据我们的经验,如果您在以测试为设计理念的架构中编写全新代码,则更容易获得自动化的关注。为很少或没有测试的现有代码编写测试充其量是一项艰巨的任务。对于刚接触敏捷和测试自动化的团队来说,这似乎几乎是不可能的。
In our experience, it’s much easier to get traction on automation if you’re writing brand new code in an architecture designed with testing in mind. Writing tests for existing code that has few or no tests is a daunting task at best. It seems virtually impossible to a team new to agile and new to test automation.
它有时是第 22 条军规。你想自动化测试,以便重构一些遗留代码,但遗留代码并不是为可测试性而设计的,所以即使在单元级别也很难自动化测试。
It is sometimes a Catch-22. You want to automate tests so you can refactor some of the legacy code, but the legacy code isn’t designed for testability, so it is hard to automate tests even at the unit level.
如果您的团队面临此类挑战并且没有预留足够的时间来集思广益如何解决它,那么就很难有效地开始自动化测试。第 14 章给出了解决这些问题的策略。
If your team faces this type of challenge and doesn’t budget plenty of time to brainstorm about how to tackle it, it’ll be tough to start automating tests effectively. Chapter 14 gives strategies to address these issues.
测试自动化对于那些从未掌握它的人来说是可怕的,甚至对于那些已经掌握它的人来说也是如此。程序员可能擅长编写生产代码,但他们可能在编写自动化测试方面经验不足。测试人员可能没有很强的编程背景,并且他们不相信自己潜在的测试自动化技能。
Test automation is scary to those who’ve never mastered it, and even to some who have. Programmers may be good at writing production code, but they might not be very experienced at writing automated tests. Testers may not have a strong programming background, and they don’t trust their potential test automation skills.
非编程测试人员经常得到的信息是,他们在敏捷世界中无能为力。我们不这么认为。没有个人测试员应该需要担心如何做自动化。这是一个团队问题,团队中通常有很多程序员可以提供帮助。诀窍是拥抱学习新想法。一次服用一天。
Non-programming testers have often gotten the message that they have nothing to offer in the agile world. We believe otherwise. No individual tester should need to worry about how to do automation. It’s a team problem, and there are usually plenty of programmers on the team who can help. The trick is to embrace learning new ideas. Take one day at a time.
当迭代进行得不顺利并且团队无法在迭代结束时完成所有的编程和测试任务时,团队成员可能会感到恐慌。我们观察到,当人们进入恐慌模式时,他们会养成舒适的旧习惯,即使这些习惯从未产生好的结果。
When iterations don’t proceed smoothly and the team can’t complete all of the programming and testing tasks by the end of an iteration, team members may panic. We’ve observed that when people go into panic mode, they fall into comfortable old habits, even if those habits never produced good results.
所以我们可能会说,“我们应该在 2 月 1 日交付。如果我们想赶上那个日期,我们就没有时间自动化任何测试了。” 我们将不得不在这段时间内完成任何可以完成的手动测试,并希望取得最好的成绩。我们以后总是可以自动化测试。”
So we may say, “We are supposed to deliver on February 1. If we want to meet that date, we don’t have time to automate any tests. We’ll have to do whatever manual tests can be done in that amount of time and hope for the best. We can always automate the tests later.”
这是灭亡之路。一些手动测试可以完成,但可能无法完成重要的手动探索性测试,这些测试可能会发现导致公司损失数十万美元销售损失的错误。然后,因为我们没有完成我们的测试自动化任务,这些任务会延续到下一个迭代,减少我们可以交付的业务价值。随着迭代的进行,情况继续恶化。
This is the road to perdition. Some manual tests can get done, but maybe not the important manual exploratory tests that would have found the bug that cost the company hundreds of thousands of dollars in lost sales. Then, because we didn’t finish with our test automation tasks, those tasks carry over to the next iteration, reducing the amount of business value we can deliver. As iterations proceed, the situation continues to deteriorate.
敏捷的全团队方法是克服自动化挑战的基础。刚接触敏捷的程序员可能习惯于交付代码而获得奖励,无论代码是否有错误,只要他们在截止日期前完成即可。测试驱动开发更多地面向设计而不是测试,因此面向业务的测试可能仍然没有进入他们的意识。它需要领导力和团队对质量的承诺,让每个人都思考如何编写、使用和运行面向技术和面向业务的测试。让整个团队参与测试自动化可能是一种文化挑战。
The agile whole-team approach is the foundation to overcoming automation challenges. Programmers who are new to agile are probably used to being rewarded for delivering code, whether it’s buggy or not, as long as they meet deadlines. Test-driven development is oriented more toward design than testing, so business-facing tests may still not enter their consciousness. It takes leadership and a team commitment to quality to get everyone thinking about how to write, use, and run both technology-facing and business-facing tests. Getting the whole team involved in test automation may be a cultural challenge.
请参阅第 3 章“文化挑战”,了解有关改变团队文化以促进敏捷实践的一些想法。
See Chapter 3, “Cultural Challenges,” for some ideas on making changes to the team culture in order to facilitate agile practices.
在下一章中,我们将展示如何使用敏捷价值观和原则来克服我们在本章中描述的一些问题。
In the next chapter, we show how to use agile values and principles to overcome some of the problems we’ve described in this chapter.
In this chapter, we analyzed some important factors related to test automation:
我们需要自动化来提供安全网,为我们提供必要的反馈,将技术债务降至最低,并帮助推动编码。
We need automation to provide a safety net, provide us with essential feedback, keep technical debt to a minimum, and help drive coding.
恐惧、缺乏知识、过去对自动化的负面经验、快速变化的代码和遗留代码是自动化的常见障碍。
Fear, lack of knowledge, negative past experiences with automation, rapidly changing code, and legacy code are among the common barriers to automation.
自动化回归测试,在自动化构建过程中运行它们,并修复缺陷的根本原因可以减少技术债务并允许可靠代码的增长。
Automating regression tests, running them in an automated build process, and fixing root causes of defects reduces technical debt and permits growth of solid code.
自动化回归测试和繁琐的手动任务可以让团队腾出时间从事更重要的工作,例如探索性测试。
Automating regression tests and tedious manual tasks frees the team for more important work, such as exploratory testing.
拥有自动化测试和自动化构建流程的团队享有更稳定的速度。
Teams with automated tests and automated build processes enjoy a more stable velocity.
如果没有自动回归测试,手动回归测试的范围将继续扩大,最终可能会被忽略。
Without automated regression tests, manual regression testing will continue to grow in scope and eventually may simply be ignored.
团队文化和历史可能使程序员更难优先考虑面向业务的测试自动化,而不是编写新功能。使用敏捷原则和价值观有助于整个团队克服测试自动化的障碍。
Team culture and history may make it harder for programmers to prioritize automation of business-facing tests than coding new features. Using agile principles and values helps the whole team overcome barriers to test automation.
在我们探索第 III 部分中的每个敏捷测试象限时,我们给出了可以帮助这些不同的测试工作取得成功的工具示例。其中许多工具用于自动化测试。正如我们在上一章中所描述的,团队在寻求成功的测试自动化过程中面临着许多障碍。更好的工具随时可用,但诀窍在于选择正确的工具并学会有效地使用它们。测试自动化需要深思熟虑的投资和渐进式改进。在本章中,我们将解释如何应用敏捷价值观和原则来启动或改进自动化工作。
As we explored each of the Agile Testing Quadrants in Part III, we gave examples of tools that can help those different testing efforts succeed. Many of those tools are for automating tests. As we described in the previous chapter, teams face plenty of obstacles in their quest for successful test automation. Better tools become available all the time, but the trick is to choose the right tools and learn to use them effectively. Test automation requires thoughtful investment and incremental improvement. In this chapter, we explain how you can apply agile values and principles to get traction in starting or improving your automation efforts.
在这里,您正在阅读本章,了解如何让您的测试自动化策略发挥作用,也许您希望找到灵丹妙药,或者您所有问题的答案。我们不想让您失望,但我们需要提前告诉您,没有灵丹妙药。没有一个答案适用于每个团队。不过不要灰心,因为我们有一些想法可以帮助您入门。
Here you are, reading this chapter on how to get your test automation strategy working, maybe hoping for that silver bullet, or an answer to all your questions. We hate to disappoint you, but we need to tell you right up front, there is no silver bullet. There is no one answer that works for every team. Don’t lose heart though, because we have some ideas to help you get started.
首先,我们建议像处理任何问题一样处理您的自动化问题。定义您要解决的问题。为了帮助您理解这一点,我们首先讨论测试自动化的一些基础知识并重新引入一些术语。
First, we suggest approaching your automation problems as you would any problem. Define the problem you are trying to solve. To help you figure that out, we first talk about some basics of test automation and reintroduce some terms.
在第三部分中,我们介绍了敏捷测试象限,并讨论了每个象限以及每个象限中测试的目的。在本节中,我们从不同的角度来看待象限。让我们仔细看看象限(见图14-1)。
In Part III, we introduced the Agile Testing Quadrants and talked about each quadrant and the purpose of the tests in each quadrant. In this section, we look at the quadrants in a different light. Let’s look carefully at the quadrants (see Figure 14-1).
您可以看到我们将支持团队的两个象限(Q1 和 Q2)标记为使用自动化。在第四象限,用于从技术角度评价产品的工具通常也需要自动化工具。在第 9 章“支持团队的面向业务的测试工具包”中,我们讨论了一些可用于自动化面向业务的测试以支持团队的工具。事实上,唯一没有标记为使用自动化的象限是象限 3——批评产品的面向业务的测试。然而,正如我们在第 10 章中所讨论的,“评价产品的面向业务的测试”,工具可能对某些测试有用。例如,自动化可以帮助设置测试数据和用户场景,以及分析记录的活动。
You can see that we’ve labeled both quadrants that support the team (Q1 and Q2) as using automation. In Quadrant 4, the tools used for critiquing the product from a technology point of view also usually require automated tools. In Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” we discussed some of the tools that can be used for automating business-facing tests in the quest for supporting the team. In fact, the only quadrant that is not labeled as using automation is Quadrant 3—the business-facing tests that critique the product. However, as we discussed in Chapter 10, “Business-Facing Tests that Critique the Product,” tools may be useful for some of that testing. For example, automation can help set up test data and user scenarios, and analyze logged activity.
使用象限来帮助您识别每个项目甚至每个迭代可能需要的不同类型的自动化工具。我们发现遍历每个象限并列出可能需要的工具清单很有帮助。假设我们要重新设计一个 UI。我们看看象限 1。如何编写测试优先的代码?我们知道如何对表示层进行单元测试吗?我们需要一个新工具来帮助解决这个问题吗?现在进入象限 2。我们需要做一些原型制作;我们应该只使用纸张,还是应该计划一个绿野仙踪类型的活动?我们将使用什么工具来创建可执行的面向业务的测试来指导开发?我们是否有需要更新或替换的回归测试脚本?我们知道我们的象限 3 活动之一将是可用性测试。这需要一些提前计划。我们可能需要工具来帮助跟踪用户的活动,以便我们可以进一步分析它们。想想象限 4,我们意识到我们有使用旧 UI 的负载测试脚本,所以我们必须预算时间来更新它们以适应新的 UI。
Use the quadrants to help you identify the different types of automation tools you might need for each project, even for each iteration. We find it helpful to go through each quadrant and make a checklist of what tools might be needed. Let’s say we’re about to redesign a UI. We look at Quadrant 1. How can it be coded test-first? Do we know how to unit test our presentation layer? Do we need a new tool to help with that? Now on to Quadrant 2. We’ll need to do some prototyping; should we just use paper, or should we plan a Wizard of Oz type activity? What tool will we use to create executable business-facing tests to guide development? Do we have regression test scripts that will need updating or replacing? We know that one of our Quadrant 3 activities will be usability testing. That takes some advance planning. We might want tools to help track the users’ activities so we can analyze them further. Thinking about Quadrant 4, we realize that we have load test scripts that use the old UI, so we have to budget time to update them for the new one.
有关 Wizard of Oz 测试的更多信息,请参阅第 8 章“支持团队的面向业务的测试”。
See Chapter 8, “Business-Facing Tests that Support the Team,” for more about Wizard of Oz testing.
正如我们在第 III 部分“使用敏捷测试象限”中强调的那样,象限的顺序与我们进行测试的顺序无关。当我们为每种类型的测试制作所需的工具清单时,我们会考虑何时进行测试,以便我们知道何时准备好自动化工具。例如,一个设计新架构的团队会计划尽快对其进行峰值测试和可扩展性测试。他们将需要在项目的第一次迭代中花时间寻找和实施性能测试工具。
As we emphasized in Part III, “Using the Agile Testing Quadrants,” the order of quadrants doesn’t relate to the order in which we do the testing. As we make our checklist of tools needed for each type of test, we think about when we want to test so we know when to have our automation tools ready. For example, a team designing a new architecture would plan to do a spike and run scalability test against it as soon as possible. They will need to spend time during the first iteration of the project finding and implementing a performance test tool.
象限帮助我们弄清楚我们可能需要什么工具,但是在不同级别有这么多不同的自动化选项,关于在哪里进行哪种类型的测试以及如何组织测试的策略是必不可少的。为了快速且经常地交付价值,我们的自动化工作需要高投资回报率。测试金字塔帮助我们优化测试投资。
The quadrants help us figure out what tools we might need, but with so many different automation options at different levels, a strategy for where to do which types of testing and how to organize the tests is essential. To deliver value quickly and often, our automation efforts need a high ROI. The test pyramid helps us optimize our test investment.
图 14-2说明了“测试自动化金字塔”。我们喜欢 Mike Cohn 介绍的版本,它显示了由面向技术的单元和组件测试组成的基础层。我们认识到许多团队会为这个想法而苦苦挣扎,因为它似乎与许多团队目前的想法相反。许多测试团队都被教导了“V”测试模型,其中组件、系统和发布测试等活动在编码活动之后按顺序进行。其他团队有一个倒金字塔,大部分测试都在功能层或表示层。
Figure 14-2 illustrates the “Test automation pyramid.” We like the version that Mike Cohn introduced, which shows the foundation layer made up of technology-facing unit and component tests. We recognize that many teams will struggle with this idea, because it seems the opposite of what many teams currently have. Many test teams have been taught the “V” model of testing, where activities such as component, system, and release testing are done in sequence after coding activities. Other teams have an inverted pyramid, with the majority of the tests in the functional or presentation layer.
敏捷测试自动化金字塔显示了三个不同的自动化测试层。最低层是支持所有其余部分的基础。它主要由健壮的单元测试和组件测试组成,这些测试是支持团队的面向技术的测试。该层代表大部分自动化测试。它们通常使用与被测系统相同的语言编写,使用 xUnit 系列工具。在一个团队掌握了 TDD 的艺术之后,这些测试是迄今为止编写起来最快、成本最低的。他们也提供最快的反馈,使他们非常有价值。在所有类型的测试中,它们的投资回报率是迄今为止最高的。
The agile test automation pyramid shows three different layers of automated tests. The lowest tier is the foundation that supports all of the rest. It’s mainly made up of robust unit tests and component tests, the technology-facing tests that support the team. This layer represents the bulk of the automated tests. They’re generally written in the same language as the system under test, using the xUnit family of tools. After a team has mastered the art of TDD, these tests are by far the quickest and least expensive to write. They provide the quickest feedback, too, making them highly valuable. They have the biggest ROI by far of any type of test.
在敏捷开发中,我们试图将尽可能多的测试推送到这一层。虽然面向业务的测试倾向于进入更高级别之一,但我们会在有意义时在单元级别实施它们。如果它们是测试,客户不必能够阅读,并且可以像单元测试一样更快地对它们进行编码,这是一个不错的选择。在单元级别也可能进行其他类型的面向技术的测试,例如性能测试。
In agile development, we try to push as many tests as possible to this layer. While business-facing tests tend to go in one of the higher levels, we implement them at the unit level when it makes sense. If they’re tests the customers don’t have to be able to read, and they can be coded much more quickly as unit tests, it’s a good option. Other types of technology-facing tests such as performance tests may also be possible at the unit level.
有关单元和组件测试的更多信息,请参阅第 7 章“支持团队的面向技术的测试”。
See Chapter 7, “Technology-Facing Tests that Support the Team” for more about unit and component tests.
金字塔的中间层是包含大多数为支持团队而编写的面向业务的自动化测试的层。这些是功能测试,可以验证我们“正在构建正确的东西”。该层中的测试可能包括“故事”测试、“验收”测试以及涵盖比单元测试层更大的功能集的测试。这些测试在 API 级别或“GUI 后面”运行,直接测试功能而无需通过 GUI。我们编写测试用例,设置输入和固定装置,将输入提供给生产代码,接受输出,并将它们与预期结果进行比较。因为这些测试绕过了表示层,所以与使用接口的测试相比,它们的编写和维护成本更低。
The middle tier in the pyramid is the layer that includes most of the automated business-facing tests written to support the team. These are the functional tests that verify that we are “building the right thing.” The tests in this layer may include “story” tests, “acceptance” tests, and tests that cover larger sets of functionality than the unit test layer. These tests operate at the API level or “behind the GUI,” testing the functionality directly without going through the GUI. We write test cases that set up inputs and fixtures that feed the inputs into the production code, accept the outputs, and compare them to expected results. Because these tests bypass the presentation layer, they are less expensive to write and maintain than tests that use the interface.
有关支持团队的面向业务的测试的更多信息,请参阅第 8 章“支持团队的面向业务的测试”。
See Chapter, 8, “Business-Facing Tests that Support the Team,” for more about business-facing tests that support the team.
我们尝试用客户可以理解的特定领域语言编写它们,因此它们比单元级测试需要更多的工作。它们通常也运行得更慢,因为每个测试比单元测试涵盖更多领域,并且可能访问数据库或其他组件。他们提供的反馈不如单元级测试快,但仍然比我们通过用户界面进行操作要快得多。因此,他们的投资回报率不如构成金字塔底层的测试高,但高于顶层。
We try to write them in a domain-specific language that the customers can understand, so they take more work than unit-level tests. They also generally run more slowly, because each test covers more ground than a unit test and may access the database or other components. The feedback they provide is not as quick as the unit-level tests, but it is still much faster than we could get operating through the user interface. Therefore, their ROI is not as high as the tests that form the base of the pyramid, but it’s higher than the top layer.
Fit 和 FitNesse 是用于金字塔中间层的工具示例。使用电子表格或其他业务友好方式定义测试用例的本土测试工具也很常见。
Fit and FitNesse are examples of tools used for the middle layer of the pyramid. Home-grown test harnesses that use spreadsheets or other business-friendly means for defining test cases are also common.
我们在第 8 章“支持团队的面向业务的测试”和第 9 章“支持团队的面向业务的测试工具包”中详细介绍了这些测试,我们在其中讨论了支持团队的面向业务的测试以及有效捕获这些测试的工具。
We have more about these tests in Chapter 8, “Business-Facing Tests that Support the Team,” and Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” where we discuss the business-facing tests that support the team and the tools that effectively capture these tests.
顶层代表应该是最小的自动化工作,因为测试通常提供最低的投资回报率。这些测试是通过GUI完成的,是真正操作和操纵表现层的。它们是在代码完成后编写的,因此通常是为了批评产品并直接进入回归套件而编写的。
The top tier represents what should be the smallest automation effort, because the tests generally provide the lowest ROI. These tests are the ones done through the GUI, the ones that actually operate and manipulate the presentation layer. They are written after the code is completed, and so are usually written to critique the product and go directly to the regression suite.
传统上,编写这些测试的成本更高,尽管有新工具可以帮助减少所需的投资。因为用户界面的组件往往会经常更改,所以这些测试比在功能或单元级别工作的测试要脆弱得多。例如,仅仅重命名 HTML 元素可能会导致测试脚本失败。与直接在生产代码上运行的金字塔较低级别的测试相比,通过用户界面进行操作也会减慢这些测试的速度。顶层的测试确实提供了重要的反馈,但是一套 GUI 测试可能需要几个小时才能运行,而不是单元级测试套件所需的几分钟。我们希望尽量减少这一层的测试数量,因此它们应该只构成金字塔的顶端。
These tests are traditionally more expensive to write, although there are new tools that help reduce the investment needed. Because components of the user interface tend to be changed often, these tests are much more brittle than tests that work at a functional or unit level. For example, just renaming HTML elements could cause a test script to fail. Operating through the user interface also slows these tests down, compared to tests in the lower levels of the pyramid that operate directly on production code. The tests in the top layer do provide important feedback, but a suite of GUI tests may take hours to run rather than the few minutes required for unit-level test suites. We want to minimize the number of tests at this layer, so they should only form the tip of the pyramid.
无论有多少自动化测试,大多数系统还需要手动测试活动,例如探索性测试和用户验收测试。我们不想忘记这些,所以我们用金字塔顶端的小云来说明它们。我们的大部分回归测试必须自动化,否则我们的手动测试也不会给我们带来良好的投资回报。
No matter how many automated tests they have, most systems also need manual testing activities, such as exploratory testing and user acceptance testing. We don’t want to forget these, so we’ve illustrated them with the little cloud at the tip of the pyramid. The bulk of our regression testing must be automated or our manual testing won’t give us a good return on investment either.
Patrick Wilson-Welsh [2008] 用“三只小猪”的比喻为测试自动化金字塔添加了一个描述性维度。底部基础层由砖砌成。测试是可靠的,并且不容易受到大灰狼的呼啸而过的影响。中间层由木棍制成。他们需要比砖层更频繁地重新排列以保持坚固。顶层的测试是用稻草做的。很难让它们留在原地,狼很容易将它们吹飞。如果我们有太多用稻草做的测试,我们将花费大量时间将它们恢复原状。
Patrick Wilson-Welsh [2008] adds a descriptive dimension to the test automation pyramid with a “three little pigs” metaphor. The bottom foundation layer is made of bricks. The tests are solid, and not vulnerable to the huffing and puffing of the Big Bad Wolf. The middle layer is made of sticks. They need rearranging more often than the brick layer to stay strong. The tests in the top layer are made of straw. It’s hard to get them to stay in place, and the wolf can easily blow them around. If we have too many tests made out of straw, we’re going to spend lots of time putting them back into shape.
大多数新的敏捷团队并不是从这个形状金字塔开始的——它通常是倒置的,是以前项目的遗留物。GUI 测试工具通常更容易学习,因此团队开始时会在其顶层“稻草”层进行大量测试。正如我们在上一章中提到的,大多数程序员为掌握单元测试自动化而必须克服的“痛苦的驼峰”意味着团队开始时可能只有几块砖。如果系统在设计时考虑到这些测试,那么在中间层自动化功能测试的装置很容易编写,所以木棍可能比积木堆得更快。随着团队掌握 TDD 和单元测试自动化,底层开始增长。当他们获得牵引力时,使用 TDD 的团队将快速构建测试金字塔的砖块基础。
Most new agile teams don’t start with this shape pyramid—it’s usually inverted, a left-over from previous projects. GUI test tools are often easier to learn, so teams start out with a lot of tests in their top “straw” layer. As we mentioned in the previous chapter, the “hump of pain” that most programmers have to overcome to master unit test automation means that the team may start out with only a few bricks. The fixtures that automate functional tests in the middle layer are easy to write if the system is designed with those tests in mind, so the sticks might pile up faster than the bricks. As teams master TDD and unit test automation, the bottom layer starts to grow. When they get traction, a team using TDD will quickly build out the brick foundation of the test pyramid.
请参阅参考书目以获取指向 Patrick Wilson-Welsh 关于“翻转测试金字塔”的讨论的链接。
See the bibliography for a link to Patrick Wilson-Welsh’s discussion of “flipping the test pyramid” right-side up.
测试金字塔是开始研究测试自动化如何帮助敏捷团队的好地方。程序员往往专注于金字塔的底部,他们需要大量的时间和培训来克服“痛苦的驼峰”并达到 TDD 自然而快速的地步。在传统团队中,测试人员通常别无选择,只能在 GUI 级别进行自动化测试。敏捷团队使用的全团队方法意味着测试人员与程序员结对并帮助他们更好地编写测试,这反过来又巩固了金字塔的基础层。因为测试驱动开发,整个团队总是在设计最大的可测试性,金字塔可以成长为正确的形状。
The testing pyramid is a good place to start looking at how test automation can help an agile team. Programmers tend to focus on the bottom of the pyramid, and they need plenty of time and training to get over the “hump of pain” and get to the point where TDD is natural and quick. In traditional teams, testers usually have no choice but to automate tests at the GUI level. The whole-team approach used by agile teams means that testers pair with programmers and help them get better at writing tests, which in turn solidifies that brick foundation layer of the pyramid. Because tests drive development, the whole team is always designing for maximum testability, and the pyramid can grow to the right shape.
程序员与测试人员配对以自动化功能级测试,填充中间层。例如,测试人员和客户可能会为 Web 服务应用程序准备 400 行的测试用例电子表格。程序员可以帮助找到一种自动化这些测试的方法。不同的团队成员可能在生成测试数据或使用 Excel 宏等工具等领域拥有专业知识,并且所有这些知识都在团队中传播。团队一起工作,找到工具、测试用例和测试数据的最佳组合。
Programmers pair with testers to automate functional-level tests, filling out the middle layer. For example, a tester and customer may prepare a 400-row spreadsheet of test cases for a web services application. The programmer can help figure out a way to automate those tests. Different team members may have expertise in areas such as generating test data or using tools such as Excel macros, and all that knowledge spreads around the team. Working together, the team finds the best combinations of tools, test cases, and test data.
让程序员参与寻找具有成本效益的方法来自动化顶层 GUI 测试有很多好处。这些努力可以让程序员更好地理解系统的“大局”,测试人员可以学习如何创建更灵活、更简单的 GUI 测试。
Involving the programmers in finding cost-effective ways to automate the top-level GUI tests has multiple benefits. These efforts may give programmers a better understanding of the system’s “big picture,” and testers can learn how to create more pliable, less straw-like GUI tests.
团队合作和共享知识越多,团队、应用程序和测试就会变得越强大。大灰狼不会有机会的。让我们首先看看我们可以自动化什么样的测试,然后看看我们甚至不应该尝试的测试。
The more a team can work together and share knowledge, the stronger the team, the application, and the tests will become. The Big Bad Wolf won’t stand a chance. Let’s start by looking at what kind of tests we can automate and then at what we shouldn’t even try.
您能想到的大多数类型的测试都受益于自动化。手动单元测试对于防止回归失败没有多大帮助,因为在每次签入之前执行一套手动测试是不切实际的。您也不能通过手动单元测试来设计测试先行的代码。当程序员不能按一下按钮快速运行测试,他们可能根本没有足够的动力来运行测试。我们可以手动测试不同的代码单元是否可以正确地协同工作,但是自动化组件测试是一个更有效的安全网。
Most types of testing you can think of benefit from automation. Manual unit tests don’t go far toward preventing regression failures, because performing a suite of manual tests before every check-in just isn’t practical. You can’t design code test-first through manual unit tests either. When programmers can’t run tests quickly at the touch of a button, they may not be motivated enough to run tests at all. We could manually test that different units of code work together correctly, but automated component tests are a much more effective safety net.
手动探索性测试是发现功能缺陷的有效方法,但如果我们没有足够的自动化面向业务的回归测试,我们可能会把所有时间都花在疯狂地试图跟上手动回归测试。让我们谈谈可以通过自动化很好地完成的所有不同类型的测试。
Manual exploratory testing is an effective way to find functional defects, but if we don’t have enough automated business-facing regression tests, we probably spend all of our time madly trying to keep up with manual regression testing. Let’s talk about all of the different kinds of testing that can be done well with automation.
要运行自动化测试,您需要某种自动化框架,允许程序员经常签入代码、对该代码运行测试并创建可部署文件。让我们先考虑一下。
To run automated tests, you need some kind of automated framework that allows programmers to check in code often, run tests on that code, and create deployable files. Let’s consider this first.
软件开发中涉及的任何繁琐或重复性任务都适合自动化。我们已经讨论了自动化构建过程的重要性。没有这个你就无法构建你的自动化测试金字塔。您的团队需要来自单元级测试的即时反馈以保持在正轨上。获取自动生成的电子邮件,列出每个签入的更改,这对测试人员有很大帮助,因为他们知道什么时候生成可以进行测试,而不必打扰程序员。
Any tedious or repetitive task involved in developing software is a candidate for automation. We’ve talked about the importance of an automated build process. You can’t build your automated test pyramid without this. Your team needs the immediate feedback from the unit-level tests to stay on track. Getting automated build emails listing every change checked in is a big help to testers because they know when a build is ready to test without having to bother the programmers.
有关构建自动化工具的示例,请参阅第 7 章“支持团队的面向技术的测试”。
See Chapter 7, “Technology-Facing Tests that Support the Team,” for examples of build automation tools.
自动化部署过程还可以加快测试速度并减少错误。事实上,珍妮特在编辑本章的那天,她搞砸了部署,因为这是一个手动过程。这很简单,但她是这个项目的新手,将文件移到了错误的地方。在 Janet 的清单上列出了需要立即完成的事情,将自动部署流程安排到位。Lisa 的团队首先实施了其持续集成和构建框架,发现这相当容易和快速,尽管它需要持续的关注和支持。其他团队,尤其是那些拥有大型复杂系统的团队,面临着更大的障碍。
An automated deployment process also speeds up testing and reduces errors. In fact, the day Janet was editing this chapter, she messed up the deployment because it was a manual process. It was pretty simple, but she was new to the project and moved the file to the wrong place. Getting an automated deployment process in place went on Janet’s list of things to get done right away. Lisa’s team implemented its continuous integration and build framework first thing, and found it fairly easy and quick to do, although it requires continual care and feeding. Other teams, especially those with large, complex systems, face much bigger hurdles.
我们与构建时间为两个小时或更长时间的团队进行了交谈。这意味着程序员在签入代码后必须等待两个小时才能验证他的签入没有破坏任何预先存在的功能。那是一个漫长的等待时间。
We’ve talked with teams who had build times of two hours or more. This meant that a programmer would have to wait for two hours after checking in code to get validation that his check-in didn’t break any preexisting functionality. That is a long time to wait.
大多数敏捷团队发现持续构建时间超过 8 到 10 分钟是不可行的。即使是 15 分钟对于等待反馈来说也太长了,因为签到将开始堆积,测试人员将等待很长时间才能获得最新、最好的构建。您能想象开发人员在接近迭代或发布周期结束时处理一个需要两个小时的构建的感觉吗?如果他们破坏了任何功能,他们将不得不再等两个小时才能了解他们是否修复了它。
Most agile teams find an ongoing build longer than eight to ten minutes to be unworkable. Even 15 minutes is much too long to wait for feedback, because check-ins will start stacking up, and testers will wait a long time to get the latest, greatest build. Can you imagine how the developers working with a build that takes two hours feel as they approach the end of an iteration or release cycle? If they break any functionality, they’ll have to wait two more hours to learn whether or not they had fixed it.
很多时候,长时间构建是访问数据库或尝试通过界面进行测试的结果。针对大型代码库运行的数千个测试会对运行构建的机器资源造成负担。对您的测试进行一些分析,看看瓶颈在哪里。例如,如果是数据库访问导致了大部分问题,请尝试模拟真实数据库并改用内存中的数据库。配置构建过程以在多台机器上分发测试。查看不同的软件是否可以帮助更好地管理资源。如果需要,请从您的团队外部聘请专家提供帮助。
Many times, long builds are the result of accessing the database or trying to test through the interface. Thousands of tests running against a large codebase can tax the resources of the machine running the build. Do some profiling of your tests and see where the bottleneck is. For example, if it is the database access that is causing most of the problems, try mocking out the real database and use an in-memory one instead. Configure the build process to distribute tests across several machines. See if different software could help manage resources better. Bring in experts from outside your team to help if needed.
加速持续集成和构建过程的关键是一次一小步。一次引入一个更改,以便您可以分别衡量每个成功并知道您在正确的轨道上。首先,您可能希望简单地删除成本最高(就时间而言)的测试以每晚运行而不是在每次构建时运行。
The key to speeding up a continuous integration and build process is to take one small step at a time. Introduce changes one at a time so that you can measure each success separately and know you are on the right track. To start with, you may want to simply remove the most costly (in terms of time) tests to run nightly instead of on every build.
快速运行的持续集成和构建过程可为任何自动化工作提供最大的投资回报率。这是每个团队需要自动化的第一件事。当它到位时,团队就有办法从自动化测试中获得快速反馈。接下来,我们看看应该自动化的不同类型的测试。
A fast-running continuous integration and build process gives the greatest ROI of any automation effort. It’s the first thing every team needs to automate. When it’s in place, the team has a way to get quick feedback from the automated tests. Next, we look at different types of tests that should be automated.
我们怎么强调自动化单元测试的重要性都不为过。如果您的程序员使用 TDD 作为一种机制来编写他们的测试,那么他们不仅在创建一个很棒的回归套件,而且还在使用它们来设计高质量、健壮的代码。如果您的团队没有自动化单元测试,那么其长期成功的机会就很小。将单元级测试自动化和持续集成作为您的首要任务。
We can’t overemphasize the importance of automating the unit tests. If your programmers are using TDD as a mechanism to write their tests, then they are not only creating a great regression suite, but they are using them to design high-quality, robust code. If your team is not automating unit tests, its chances of long-term success are slim. Make unit-level test automation and continuous integration your first priority.
第 7 章,“支持团队的面向技术的测试”,详细介绍了一些可以使用的工具。
Chapter 7, “Technology-Facing Tests that Support the Team,” goes into detail about some of the tools that can be used.
使用某种形式的自动化最容易测试 API 或 Web 服务应用程序。Janet 所在的团队成功地使用 Ruby 读取包含输入变量的所有排列和组合的电子表格,并将输出与存储在电子表格中的预期结果进行比较。这些数据驱动的测试易于编写和维护。
Testing an API or web services application is easiest using some form of automation. Janet has been on teams that have successfully used Ruby to read in a spreadsheet with all of the permutations and combinations of input variables and compare the outputs with the expected results stored in the spreadsheets. These data-driven tests are easy to write and maintain.
Janet 的一位客户使用 Ruby 的 IRB(交互式 Ruby Shell)功能来测试 Web 服务以进行验收测试。该团队愿意与客户团队共享其脚本,但业务测试人员更愿意观察如果即时更改输入会发生什么。以半自动方式交互式运行测试允许这样做。
One customer of Janet’s used Ruby’s IRB (Interactive Ruby Shell) feature to test the web services for acceptance tests. The team was willing to share its scripts with the customer team, but the business testers preferred to watch to see what happened if inputs were changed on the fly. Running tests interactively in a semiautomated manner allowed that.
GUI 背后的测试比 GUI 本身的测试更容易实现自动化。因为测试不受表示层更改的影响,并且在更稳定的业务逻辑代码上工作,所以它们更稳定。用于此类测试的工具通常使用表格或电子表格以声明性格式编写测试。获取生产代码以对测试输入进行操作并返回结果的固定装置通常可以快速编写。这是编写面向业务的测试的主要领域,客户和推动开发的开发人员都可以理解。
Testing behind the GUI is easier to automate than testing the GUI itself. Because the tests aren’t affected by changes to the presentation layer and work on more stable business logic code, they’re more stable. Tools for this type of testing typically provide for writing tests in a declarative format, using tables or spreadsheets. The fixtures that get the production code to operate on the test inputs and return the results can generally be written quickly. This is a prime area for writing business-facing tests, understandable to both customers and developers that drive development.
有关特定工具示例,请参阅第 9 章“支持团队的面向业务的测试工具包”。
See Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” for specific tool examples.
即使是只有很少或没有业务逻辑的瘦 GUI 也需要进行测试。敏捷开发的快速步伐,每次迭代都交付新功能,要求在 GUI 级别对大多数项目进行一些自动化回归测试。工具选择是成功实现 GUI 自动化的关键。自动化脚本需要灵活且易于维护。当使用良好的编码实践开发框架时,Janet 非常成功地使用了 Ruby 和 Watir,就好像它是一个生产应用程序一样。开发库的时间被投入了,这样代码中就没有太多的返工或重复,并且可以在一个地方进行所需的更改。使代码易于维护可以增加这些测试的投资回报率。
Even a thin GUI with little or no business logic needs to be tested. The fast pace of agile development, delivering new functionality each iteration, mandates some automated regression tests at the GUI level for most projects. Tool selection is key for successful GUI automation. The automated scripts need to be flexible and easy to maintain. Janet has used Ruby and Watir very successfully when the framework was developed using good coding practices, just as if it were a production application. Time was put into developing the libraries so that there was not a lot of rework or duplication in the code, and changes needed could be made in one place. Making the code easy to maintain increased the ROI on these tests.
这里有一点关于可测试性——确保程序员命名他们的对象或为他们分配 ID。如果它们依赖于系统生成的标识符,那么每次将新对象添加到页面时,ID 都会发生变化,从而需要对测试进行更改。
A point about testability here—make sure the programmers name their objects or assign IDs to them. If they rely on system-generated identifiers, then every time a new object is added to the page, the IDs will change, requiring changes to the tests.
有关 GUI 测试框架的示例,请参阅第 9 章“支持团队的面向业务的测试工具包”。
See Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” for examples of GUI test frameworks.
只对实际界面进行测试。检查一些事情,比如确保按钮真正起作用并做它们应该做的事情。不要试图测试业务功能。其他可以轻松自动化的测试类型是链接检查器。无需有人手动检查每个页面上的每个链接以确保他们访问了正确的页面。寻找唾手可得的果实,先将容易实现自动化的事情自动化,这样您就会有更多时间应对更大的挑战。
Keep the tests to just the actual interface. Check things like making sure the buttons really work and do what they are supposed to. Don’t try to try to test business functionality. Other types of tests that can be automated easily are link checkers. There is no need for someone to manually go through every link on every page to make sure they hit the right page. Look for the low-hanging fruit, automate the things that are simple to automate first, and you’ll have more time for the bigger challenges.
没有自动化就无法完成某些类型的测试。手动负载测试通常不可行或不准确,尽管我们都曾经或多次尝试过。性能测试既需要监控工具,也需要一种在被测系统中驱动操作的方法。如果没有一些工具框架,您无法生成高容量攻击来验证网站是否可以被黑客入侵或是否可以处理大量负载。
Some types of testing can’t be done without automation. Manual load tests aren’t usually feasible or accurate, although we’ve all tried it at one time or another. Performance testing requires both monitoring tools and a way to drive actions in the system under test. You can’t generate a high-volume attack to verify whether a website can be hacked or can handle a large load without some tool framework.
有关负载测试自动化工具的示例,请参阅第 11 章“使用面向技术的测试来评价产品”。
See Chapter 11, “Critiquing the Product Using Technology-Facing Tests,” for examples of load test automation tools.
如果您首先解析文件并以人类可读的格式显示它,那么直观地检查系统进程输出的 ASCII 文件会容易得多。用于比较输出文件以确保没有进行无意更改的脚本比尝试手动比较它们更快、更准确。文件比较工具比比皆是,从免费的 diff 到 WinDiff 等专有工具。源代码管理工具和 IDE 都有自己的内置比较工具。这些是每个测试人员工具箱中必不可少的项目。在为数据仓库或数据迁移项目进行测试时,不要忘记创建用于比较数据库表的脚本。
Visually checking an ASCII file output by a system process is much easier if you first parse the file and display it in a human-readable format. A script to compare output files to make sure no unintentional changes were made is a lot faster and more accurate than trying to compare them manually. File comparison tools abound, ranging from the free diff to proprietary tools such as WinDiff. Source code management tools, and IDEs have their own built-in comparison tools. These are essential items in every tester’s toolbox. Don’t forget about creating scripts for comparing database tables when doing testing for your data warehouse or data migration projects.
在第 7 章“支持团队的面向技术的测试”中阅读有关源代码管理工具和 IDE 的更多信息。
Read more about source code management tools and IDEs in Chapter 7, “Technology-Facing Tests that Support the Team.”
当我们与客户合作以更好地了解业务并了解对他们有价值的东西时,我们可能会看到将他们的某些任务自动化的机会。丽莎的公司需要向所有客户邮寄几份带有求职信的表格。程序员不仅可以生成表格,还可以将它们与求职信连接起来,大大加快了邮寄速度。Lisa 的测试伙伴 Mike Busse 编写了一个电子表格宏来进行复杂的资金分配计算,而退休计划管理员一直在手动进行这些计算。许多手动清单可以用自动化脚本代替。自动化不仅仅用于测试。
As we work with our customers to better understand the business and learn what’s valuable to them, we might see opportunities to automate some of their tasks. Lisa’s company needed to mail several forms with a cover letter to all of their clients. The programmers could not only generate the forms but could also concatenate them with the cover letter and greatly speed up the mailing effort. Lisa’s fellow tester, Mike Busse, wrote a spreadsheet macro to do complex calculations for allocating funds that the retirement plan administrators had been doing manually. A lot of manual checklists can be replaced with an automated script. Automation isn’t just for testing.
另一个有用的自动化领域是数据创建或设置。如果您经常设置数据,请自动化该过程。通常,我们需要多次重复某件事才能重现错误。如果这可以自动化,你将保证每次都有相同的结果。
Another useful area for automation is data creation or setup. If you are constantly setting up your data, automate the process. Often, we need to repeat something multiple times to be able to recreate a bug. If that can be automated, you will be guaranteed to have the same results each time.
清理测试数据与生成测试数据一样重要。您的数据创建工具包应该包括拆除测试数据的方法,这样它就不会影响不同的测试或阻止重新运行相同的测试。
Cleaning up test data is as important as generating it. Your data creation toolkit should include ways to tear down the test data so it doesn’t affect a different test or prevent rerunning the same test.
我们已经研究了需要或至少有用自动化的主要领域。我们的意见是,每当您需要进行测试或一些与测试相关的活动时,首先要确定是否可以通过自动化来辅助。在某些情况下,自动化并不合适。让我们看看其中的一些。
We’ve looked at major areas where automation is required or at least useful. Our opinion is that whenever you need to do a test or some testing-related activity, first decide whether it can be aided by automation. In some cases, automation won’t be appropriate. Let’s look at some of those.
有些测试需要人的眼睛、耳朵和智慧。可用性和探索性测试属于这一类。其他可能无法证明自动化投资合理的测试是一次性测试和永远不会失败的测试。
Some testing needs human eyes, ears, and intelligence. Usability and exploratory testing are two that fall into that category. Other tests that may not justify the automation investment are one-off tests and those that will never fail.
真正的可用性测试需要有人实际使用该软件。自动化可能有助于设置场景以随后检查可用性。观察用户的行为,向他们汇报他们的体验,并判断结果是了解软件可用性方面无法自动化的人的工作。记录用户操作有助于可用性测试。
Real usability testing requires someone to actually use the software. Automation might be helpful in setting up scenarios to subsequently examine for usability. Observing users in action, debriefing them on their experiences, and judging the results is a job for a person who understands that usability aspects of software cannot be automated. Logging user actions is helpful for usability testing.
我们在第 10 章“评价产品的面向业务的测试”中讨论了一些日志记录和监控工具。
We discuss some logging and monitoring tools in Chapter 10, “Business-Facing Tests that Critique the Product.”
可以通过自动化测试来确保 GUI 永不更改,但您需要问问自己是否值得为此付出代价。你真的关心一个按钮改变了一个像素的位置吗?结果证明努力是正确的吗?我们认为您不应该自动化“外观和感觉”测试,因为自动化脚本只能查找您告诉它要查看的内容。自动化会错过会跳到人类身上的视觉问题。
It is possible to automate tests that make sure the GUI never changes, but you need to ask yourself whether it’s worth the cost. Do you really care that a button has changed positions by one pixel? Do the results justify the effort? We don’t think you should automate “look and feel” testing, because an automated script can only look for what you tell it to see. Automation would miss visual problems that would jump out at a human.
同样,探索性测试可以通过脚本来加速以创建测试数据并跳过一些设置步骤,但它需要熟练的测试人员来设计和执行测试。探索性测试的一个主要目标是通过实践更多地了解产品,然后使用该信息改进未来的开发。自动化脚本不会为您做这些。但是,正如我们之前所说,如果没有大量其他自动化测试,您将没有时间进行探索性测试。
Similarly, exploratory testing may be speeded up with scripts to create test data and jump through some setup steps, but it requires a skilled tester to design and execute the tests. One major goal of exploratory testing is to learn more about the product by doing, and then use that information to improve future development. Automated scripts won’t do that for you. However, as we’ve said before, you won’t have time for exploratory testing without a lot of other automated tests.
请参阅第 10 章,“评价产品的面向业务的测试”,了解更多关于探索性测试和可以促进它的工具。
See Chapter 10, “Business-Facing Tests that Critique the Product,” for more on exploratory testing and tools that can facilitate it.
我们听说过这样一种说法,即永远不会失败的测试不需要自动化。如果一个需求如此明显以至于只有一种方法可以实现它,并且没有程序员会在不知道它应该做什么的情况下查看该代码,那么有人在该代码中引入缺陷的可能性几乎为零。假设我们有一个带有地址字段的表单。我们是否需要自动回归测试来验证不需要第二条街道地址行?在我们手动验证之后,以后有人不小心将其更改为必填字段的可能性有多大?即使有人这样做,也不会是灾难性的事件。其他人会注意到它,人们可以轻松解决它,直到它被修复。
We’ve heard an argument that tests that will never fail don’t need to be automated. If a requirement is so obvious that there’s only one way to implement it, and no programmer will ever look at that code later without knowing exactly what it should do, the chances of someone introducing a defect in that code are next to nothing. Let’s say we have a form with address fields. Do we need an automated regression test to verify that the second street address line is not required? After we’ve verified it manually, how likely is it that someone will accidentally change it to a required field later? Even if someone did, it wouldn’t be a catastrophic event. Someone else would notice it and people could work around it easily until it was fixed.
话又说回来,对它的测试很容易包括在内。诸如复制/粘贴错误之类的程序员技巧一直在发生。如果您觉得一次性手动测试就可以完成工作并且未来失败的风险不足以证明自动化回归测试是合理的,请不要将它们自动化。如果您的决定被证明是错误的,您以后将有另一个机会使它们自动化。如果您不确定,并且自动化并不难,那就去做吧。
Then again, a test for it would be easy to include. And programmer tricks such as copy/paste errors happen all the time. If you feel comfortable that one-time manual testing does the job and that the risk of future failures doesn’t justify automating regression tests, don’t automate them. If your decision turns out to be wrong, you’ll get another chance to automate them later. If you aren’t sure, and it’s not terribly difficult to automate, go for it.
如果您正在测试一个生命攸关的系统,即使是非常小的回归失败风险也太大了。使用风险分析来帮助决定哪些测试应该自动化。
If you’re testing a life-critical system, even a very small risk of a regression failure is too much. Use risk analysis to help decide what tests should be automated.
有关风险分析及其与测试的关系的更多信息,请参阅第 18 章“编码和测试”。
See Chapter 18, “Coding and Testing,” for more about risk analysis and how it relates to testing.
大多数时候,手动执行一次性测试就足够了。如果自动化测试没有回报,为什么要这么做?有时自动化对于一次性测试来说是值得的。
Most times, manually executing a one-off test is sufficient. If automating a test doesn’t have payoff, why do it? Sometimes automation is worth doing for a one-off test.
繁琐的任务可能值得自动化,即使您不经常这样做。权衡自动化成本与手动执行测试所消耗的宝贵时间。如果它很容易手动完成,而自动化不会很快,那就保持手动。
Tedious tasks may be worth automating, even if you don’t do them often. Weigh the automation cost against the amount of valuable time eaten up by manually doing the test. If it’s easy to do manually, and automating wouldn’t be quick, just keep it manual.
如果代码不是以测试为先编写的,或者至少没有考虑到测试自动化,那么自动化就更难了。较旧的系统往往属于这一类,但毫无疑问,仍在生产大量具有相同不可测试特性的新代码。
When code isn’t written test-first, or at least with test automation in mind, it’s much harder to automate. Older systems tend to fall into this category, but no doubt plenty of new code with the same untestable characteristics is still being produced.
如果您面临处理尚未进行自动化测试的现有代码,那么您将面临一场艰苦的战斗,但这是一场可以取胜的战斗。遗留代码可能有 I/O、数据库访问以及业务逻辑和表示代码交织在一起。可能不清楚在何处挂钩代码以自动执行测试。您如何开始在这样的系统上进行自动化测试?您当然不能计划自动化 GUI 下的所有内容,因为大部分逻辑都在表示层中。
If you’re faced with working on existing code that doesn’t already have automated tests, you’re in for an uphill battle, but a winnable one. Legacy code may have I/O, database access, as well as business logic and presentation code intertwined. It may not be clear where to hook into the code to automate a test. How do you get started automating tests on such a system? You certainly can’t plan on automating everything below the GUI, because much of the logic is in the presentation layer.
至少有几种不同的方法行之有效。我们在第 13 章“我们为什么要自动化测试以及是什么阻碍了我们”中谈到的“痛苦的驼峰”令人生畏,但它是可以克服的,然后测试自动化将变得容易得多。Michael Feathers 的Working Effectively With Legacy Code [2004] 解释了如何围绕现有代码库构建测试工具并重构它们以适应自动化。即使使用遗留代码,您也可以编写测试来防止引入新问题。这种方法甚至适用于缺乏结构或非面向对象的系统。
There are at least a couple of different approaches that work well. The “hump of pain” we talked about in Chapter 13, “Why We Want to Automate Tests and What Holds Us Back,” is intimidating, but it can be overcome, and then test automation will become much easier. Michael Feathers’ Working Effectively With Legacy Code [2004] explains how to build a test harness around existing code bases and refactor them to accommodate automation. Even with legacy code, you can write tests to protect against introducing new problems. This approach can work even on systems that lack structure or aren’t object-oriented.
Lisa 的团队决定采用一种不同但同样有效的方法。团队成员开始通过在新的测试友好架构中编写所有新功能来“扼杀”遗留代码。他们逐渐用测试优先编写的代码替换所有旧代码。当他们确实处理旧代码以修复错误时,或者在旧代码需要更新的情况下,他们只需为他们更改的所有代码添加单元测试。GUI 冒烟测试套件涵盖了没有单元测试的遗留系统其余部分的关键功能。
Lisa’s team decided on a different but equally effective approach. The team members started “strangling” the legacy code by writing all new features in a new test-friendly architecture. They’re gradually replacing all of the old code with code written test-first. When they do work on old code to fix bugs, or in the cases where the old code needs updating, they simply add unit tests for all of the code they change. A GUI smoke test suite covers the critical functions of the rest of the legacy system that has no unit tests.
第 7 章,“支持团队的面向技术的测试”,详细介绍了处理遗留代码的不同敏捷方法。
Chapter 7, “Technology-Facing Tests that Support the Team,” goes into more detail about different agile approaches to legacy code.
与任何自动化项目一样,一次处理一个难以自动化的代码,并首先解决风险最高的区域。解决可测试性问题并找到编写单元级测试的方法。努力会有回报。
As with any automation project, approach the hard-to-automate code one piece at a time, and address the highest risk areas first. Solve the testability problem and find a way to write unit-level tests. The effort will pay off.
一个简单的、循序渐进的方法听起来与自动化策略不相容,但在敏捷测试中,我们试图首先理解问题。决定从哪里以及如何开始自动化需要一些思考和讨论。当您的团队着眼于测试挑战时,您将需要考虑自动化适合的地方。在开始搜索特定的自动化工具之前,您需要确定您的需求。
A simple, step-by-step approach sounds incompatible with an automation strategy, but in agile testing we try to understand the problem first. Deciding where and how to start with automation requires a bit of thought and discussion. As your team looks at testing challenges, you’ll need to consider where automation is appropriate. Before you start searching for a particular automation tool, you’ll want to identify your requirements.
您需要了解您要解决的问题是什么。你想自动化什么?例如,如果您没有任何类型的测试自动化,并且您开始购买昂贵的商业测试工具并认为它可以自动执行您所有的功能测试,那么您可能从错误的地方开始。
You need to understand what problem you are trying to solve. What are you trying to automate? For example, if you have no test automation of any kind, and you start by buying an expensive commercial test tool thinking it will automate all your functional tests, you may be starting in the wrong place.
我们建议您从头开始。寻找你最大的收获。最划算的绝对是程序员可以做的单元测试。与其从测试金字塔的顶部开始,不如从底部开始,确保基础知识到位。你还需要考虑您需要自动化的不同类型的测试,以及何时需要准备好可用的工具。
We suggest you start at the beginning. Look for your biggest gain. The biggest bang for the buck is definitely the unit tests that the programmers can do. Instead of starting at the top of the test pyramid, you may want to start at the bottom, making sure that the basics are in place. You also need to consider the different types of tests you need to automate, and when you’ll need to have tools ready to use.
在本节中,我们假设您已经对第 1 象限的单元和组件测试进行了自动化,并希望在第 2 和第 3 象限中实现面向业务的测试自动化,或者使您的第 4 象限面向技术的测试对产品进行评估。我们将帮助您设计一个好的策略来构建您的自动化资源。
In this section, we assume you have automated Quadrant 1 unit and component tests in place, and are looking to automate your business-facing tests in Quadrants 2 and 3, or your Quadrant 4 technology-facing tests that critique the product. We’ll help you design a good strategy for building your automation resources.
想想你团队的技能和经验。谁需要自动化,为什么?你想达到什么目标?了解其中一些问题可能会影响您对工具的选择以及您付出的努力。本章末尾有一节是关于评估工具的。
Think about the skills and experience on your team. Who needs the automation, and why? What goals are you trying to achieve? Understanding some of these issues may affect your choice of tools and what effort you expend. There is a section on evaluating tools at the end of this chapter.
自动化很可怕,尤其是如果您是从头开始,那么我们从哪里开始呢?
Automation is scary, especially if you’re starting from scratch, so where do we begin?
要弄清楚下一步应该将自动化工作的重点放在何处,请询问您的团队,“最痛苦的地方是什么?” 或者,对于某些团队来说,“最无聊的地方是什么?” 你甚至可以部署代码来测试它吗?团队成员对更改代码是否有信心,或者他们是否缺乏自动化测试的任何安全网?也许您的团队成员更高级,掌握了 TDD,并且拥有全套单元测试。但是他们没有一个好的框架来指定面向业务的测试,或者不能完全掌握自动化测试的方法。也许您确实有一些 GUI 测试,但它们非常慢并且维护成本很高。
To figure out where to focus your automation efforts next, ask your team, “What’s the greatest area of pain?” or, for some teams, “What’s the greatest area of boredom?” Can you even get code deployed in order to test it? Do team members feel confident about changing the code, or do they lack any safety net of automated tests? Maybe your team members are more advanced, have mastered TDD, and have a full suite of unit tests. But they don’t have a good framework for specifying business-facing tests, or can’t quite get a handle on automating them. Perhaps you do have some GUI tests, but they’re extremely slow and are costing a lot to maintain.
无论哪里受到的伤害最大,那就是您开始自动化工作的地方。例如,如果您的团队甚至难以交付可部署的代码,您就需要实施自动化构建流程。没有什么比在等待一些代码测试时摆弄拇指更糟糕的了。
Wherever it hurts the most, that’s the place to start your automation efforts. For example, if your team is struggling to even deliver deployable code, you need to implement an automated build process. Nothing’s worse than twiddling your thumbs while you wait for some code to test.
但是,如果性能使您的组织处于危险之中,则性能测试必须是重中之重。这又回到了了解您要解决的问题上。风险分析是你的朋友。
But, if performance puts the existence of your organization in danger, performance testing has to be the top priority. It’s back to understanding what problem you are trying to solve. Risk analysis is your friend here.
第 18 章“编码和测试”包含有关风险分析的简单方法的更多信息。
Chapter 18, “Coding and Testing,” has more information on a simple approach to risk analysis.
除非有其他良好的开发实践,否则测试自动化不会带来回报。运行一套健壮的单元测试的持续集成是实现其他测试自动化的第一步。为可维护性和良好设计而不断重构的代码将有助于提高自动化的投资回报率。没有良好的单元测试覆盖率,就不可能进行重构。这些开发实践也需要应用于自动化功能测试脚本。
Test automation won’t pay off unless other good development practices are in place. Continuous integration running a robust suite of unit tests is a first step toward automating other tests. Code that’s continually refactored for maintainability and good design will help increase the ROI on automation. Refactoring can’t happen without that good unit test coverage. These development practices also need to be applied to the automated functional test scripts.
虽然我们建议一次掌握一种工具,但不要对任何一种工具期望过高。为每个需要使用正确的工具。最适合单元测试的工具可能适合也可能不适合自动化功能测试。GUI、负载、性能和安全测试可能各自需要一个或多个不同的工具。
While we recommend mastering one tool at a time, don’t expect too much out of any one tool. Use the right tool for each need. The tool that works best for unit tests may or may not be appropriate to automate functional tests. GUI, load, performance, and security testing may each require a different tool or tools.
Mike Cohn 的测试金字塔概念(参见图 14-2)帮助我们的团队将他们的自动化工作放在他们最擅长的地方。我们希望最大化具有最佳投资回报率的测试。如果系统架构是为可测试性而设计的,那么测试自动化的成本就会更低,尤其是在单元级别。
Mike Cohn’s test pyramid concept (see Figure 14-2) has helped our teams put their automation efforts where they do the most good. We want to maximize the tests that have the best ROI. If the system architecture is designed for testability, test automation will be less expensive, especially at the unit level.
通过用户界面进行的测试通常具有最低的投资回报率,因为它们的维护成本很高,但我们确实需要它们。它们构成了我们金字塔的一小部分 我们可能会选择将其中一些测试自动化,但大多数 GUI 测试都是用业务术语定义的,可能最好保留为人机交互测试(即手动测试)。
Tests that go through the user interface usually have the lowest ROI, because they’re expensive to maintain, but we do need them. They make up the small tip of our pyramid We may choose to automate some of these tests, but the majority of GUI tests are defined in business terms and probably are best left as human interaction tests (i.e., manual tests).
中间层代表直接与生产代码一起工作的功能测试,中间没有 GUI 或其他层。虽然它们的自动化成本不如单元级测试那么低,而且提供反馈的速度也稍慢一些,但正确的工具可以让它们获得良好的投资回报率。这些测试可以用业务专家理解的语言编写,这一事实增加了它们的价值。
The middle layer represents the functional tests that work directly with production code, without a GUI or other layer in between. While they’re not as inexpensive to automate as unit-level tests, and provide feedback a bit more slowly, the right tools allow them to have a good ROI. The fact that these tests can be written in a language the business experts understand adds to their value.
应用程序中有许多不同的层可以独立测试。在他的书xUnit 测试模式[2007] 中,Gerard Meszaros 将此称为层测试模式。他告诫说,当尝试单独测试应用程序的所有层时,我们仍然必须验证这些层是否正确连接,这可能需要至少通过表示层对业务逻辑进行一次测试。
There are many different layers in the application that can be tested independently. In his book xUnit Test Patterns [2007], Gerard Meszaros refers to this as the Layer Test pattern. He cautions that when trying to test all of the layers of the application separately, we still have to verify that the layers are hooked up correctly, and this may require at least one test of the business logic through the presentation layer.
在评估自动化工作的回报时,请考虑不太具体的方面,例如该工具是否促进了团队之间的协作技术和客户团队。编写测试的一个主要原因是帮助指导开发。如果编写自动化验收测试的过程导致对业务需求的透彻理解,那将是巨大的回报,即使测试以后再也找不到一个单一的回归错误。
When evaluating the payback of your automation efforts, consider less tangible aspects such as whether the tool promoted collaboration between the technical and customer teams. A primary reason to write tests is to help guide development. If the process of writing your automated acceptance tests results in a thorough understanding of business requirements, that’s plenty of payback, even if the tests never find a single regression bug later on.
想一想您一生中编写的所有手动测试脚本。难道你不希望所有这些都自动化吗?你的生活不会轻松很多吗?我们认为所有脚本测试都应该自动化。让我们开始转换那些手动脚本测试。
Think about all of the manual test scripts you’ve written in your life. Don’t you just wish those all would have been automated? Wouldn’t your life have been a lot easier? We believe that all scripted tests should be automated. Let’s get started converting those manual scripted tests.
开始后,自动化测试会非常容易。例如,当您有一个工作的 FitNesse 夹具时,添加更多测试用例几乎不需要付出任何努力。当您有很多不同的排列要测试时,这很棒。与手动完成所有测试相比,您可能会测试更多的条件。当 Lisa 的团队成员重写他们退休计划的贷款系统时,他们可以通过 FitNesse 测试来测试数百种不同的贷款支付处理可能性。如果同一天处理三笔贷款付款会怎样?如果有人三个月没有支付任何款项,然后又支付了一大笔款项,那么利息的计算和应用是否正确?很容易编写自动化测试来找出答案。
After you get started, it can be quite easy to automate tests. For example, when you have a working FitNesse fixture, adding more test cases requires little effort. This is great when you have a lot of different permutations to test. You’ll probably test more conditions than you would if all your testing was done manually. When Lisa’s team members rewrote their retirement plan’s loan system, they could test hundreds of different possibilities for loan payment processing via FitNesse tests. What happens when three loan payments are processed on the same day? If someone doesn’t make any payments for three months, and then sends in a large payment, is the interest calculated and applied correctly? It was easy to write automated tests to find out.
这是一个很大的优势,但也有不利的一面。现在团队有几十个,甚至上百个测试用例需要维护。如果一年后计算贷款利息金额的规则发生变化怎么办?这可能需要更新每个测试。如果您的测试工具不能轻易适应对现有测试的更改,那么您的大型自动化测试套件可能会变得令人头疼。
That’s a great advantage, but it has a down side. Now the team has dozens, or even hundreds, of test cases to maintain. What if the rules about calculating the amount of interest for a loan payment change a year from now? This could require updating every test. If your test tool doesn’t easily accommodate making changes to existing tests, your big suite of automated tests can turn into a headache.
端到端测试特别难以自动化,因为随着业务规则的变化,它们最有可能需要维护。我们如何平衡自动化的需求与成本?
End-to-end tests are particularly tricky to automate because they have the most potential to need maintenance as business rules change. How do we balance the need for automation with the cost?
请记住从您正在测试的功能的薄片或钢丝开始。就像程序员接近编码一样接近自动化。让一小部分钢丝工作,然后继续下一个。在你覆盖了整个薄片之后,回去把它充实起来。
Remember to start with the thin slice or steel thread of the feature you’re testing. Approach automation just as programmers approach coding. Get one small unit of the steel thread working, and then move on to the next. After you’ve covered the whole thin slice, go back and flesh it out.
第 8 章,“支持团队的面向业务的测试”,详细解释了薄片。
Chapter 8, “Business-Facing Tests that Support the Team,” explains more about thin slices.
仔细选择您的测试模式。自动化您需要的所有测试用例,但仅此而已,并在您可以的最低级别自动化它们。将每个测试用例的范围限制为一个测试条件或一个业务规则。了解测试的目的。避免测试之间的依赖关系,因为它们会迅速增加复杂性和维护费用。
Choose your test pattern thoughtfully. Automate all of the test cases you need, but no more, and automate them at the lowest level that you can. Limit the scope of each test case to one test condition or one business rule. Understand the purpose of the test. Avoid dependencies between tests, because they quickly increase complexity and maintenance expense.
有关有效测试设计的更多信息,请参阅第 9 章“支持团队的面向业务的测试工具包”。
See Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” for more information about effective test design.
正如我们之前提到的,自动化测试的级别越低,投资回报率就越高。尽可能将测试自动化推向金字塔的下端。如果您的单元测试和代码集成测试覆盖率很高,则不需要自动化那么多的功能测试。有了较低级别的可靠覆盖,手动进行端到端测试来验证系统的行为可能就足够了。使用风险分析来帮助您做出决定。
As we’ve mentioned before, the lower the level at which you automate a test, the better the ROI. Push test automation as far down the pyramid as you can. If you have good coverage in your unit and code integration tests, you don’t need to automate as many functional tests. With solid coverage at the lower levels, it might be enough to do end-to-end tests manually to verify the system’s behavior. Use risk analysis to help you decide.
用户界面确实需要测试。在某些情况下,GUI 级别的测试自动化至关重要。也许您的团队正在使用第三方 GUI 控件,而您不确定它们的行为方式。如果您的风险和 ROI 分析支持 GUI 级别的大量自动化,请进行投资。
The user interface does need to be tested. In some situations, test automation at the GUI level is critical. Perhaps your team is using third-party GUI controls, and you aren’t sure how they will behave. If your risk and ROI analysis supports a lot of automation at the GUI level, make the investment.
如果您在更高级别进行自动化,请不要过火并使系统中的每条可能路径都自动化。您不必将在开发阶段创建的每个自动化测试都保留在回归套件中;考虑构建时间和发现缺陷的机会之间的权衡。集中精力覆盖单元、代码集成和功能级别的代码中的每条重要路径。你会得到更好的回报。
If you do automate at the higher levels, don’t go overboard and automate every possible path through the system. You don’t have to keep every automated test created during the development phase in the regression suite; consider tradeoffs of build time and the chance of finding defects. Focus your efforts on covering every important path through the code at the unit, code integration, and functional levels. You’ll get a much better payback.
取得平衡不是敏捷原则,它只是常识。您现在需要一个足够好的解决方案,但它不一定是完美的。该工具是否提供您现在需要的结果?它是否为将其用于自动化所需的资源提供了足够的回报?如果是这样,请继续使用它,并在以后安排时间寻找替代方案。随着时间的推移,您可以改进您的自动化框架。最重要的因素是您的自动化工具现在是否适合您的特定情况。
Striking a balance isn’t an agile principle, it’s just common sense. You need a good-enough solution right now, but it doesn’t have to be perfect. Does the tool provide the results you need right now? Does it provide an adequate return on the resources needed to use it for automation? If so, go ahead and use it, and budget time later to look for alternatives. You can improve your automation framework over time. The most important factor is whether your automation tools fit your particular situation right now.
不要反过来想,“好吧,我们可以用这个记录工具生成一堆脚本,立即完成测试,然后重构脚本以使其可维护。” 虽然您不需要一直寻找完美的自动化解决方案,但您确实需要一个不增加您团队的技术债务。在“它找到我们需要了解的错误并且维护成本不高”和“这是我们能找到的最优雅、最酷的解决方案”之间找到平衡。
Don’t slide the other way, and think, “OK, we can generate a bunch of scripts with this record tool, get our immediate testing done, and refactor the scripts later to make them maintainable.” While you don’t need to keep searching for the perfectly ideal automation solution, you do need a solution that doesn’t add to your team’s technical debt. Find a balance between “It finds the bugs we need to know about and doesn’t cost too much to maintain” and “This is the most elegant and cool solution we can find.”
很酷的是,我们有这么多工具可以帮助我们解决自动化问题。不要追求比你需要的更复杂的东西。Lisa 的同事发现,从数据库中检索数据并独立于系统执行计算的电子表格是一种强大的工具,既可用于推动开发,也可用于验证应用程序的计算。
It’s cool that we have so many tools available to help us solve our automation problems. Don’t go for more sophistication than you need. Lisa’s coworkers have found that a spreadsheet that retrieves data from the database and performs calculations independently of the system is a powerful tool, both for driving development and for verifying the application’s calculations.
我们通常尽量减少 GUI 层的测试自动化,但在某些情况下,更多的 GUI 自动化是合适的。如果用户在 X 处进行了更改,还有什么变化?有些问题仅在 GUI 级别表现出来。当退休计划参与者要求从他们的账户中分配资金时,Lisa 测试了一个解决后端问题的错误修复。更改被单元测试包围,但它是一个 GUI 回归测试,当分发表单无法根据请求弹出时失败。没有人预料到后端更改会影响 GUI,因此他们可能不会费心手动测试它。这也是您需要 GUI 回归测试的原因。
We usually minimize test automation at the GUI layer, but there are situations where more GUI automation is appropriate. If the user makes a change at X, what else changes? Some problems only manifest themselves at the GUI level. Lisa tested a bug fix that addressed a back-end problem when retirement plan participants requested a distribution of money from their accounts. The change was surrounded by unit tests, but it was a GUI regression test that failed when the distribution form failed to pop up upon request. Nobody anticipated that a back-end change could affect the GUI, so they probably wouldn’t have bothered to test it manually. That’s why you need GUI regression tests, too.
我们已经讨论了录制/回放工具的一些缺点,但它们适用于正确的情况。您可能有充分的理由使用记录/回放工具:也许您的遗留代码已经在该工具中创建了一套自动化测试,您的团队在该工具方面拥有丰富的专业知识,或者您的管理层希望您将其用于不管什么原因。您可以使用录制的脚本作为起点,然后将脚本分解为模块,在适当的地方用参数替换硬编码数据,并使用模块作为构建块来组装测试。即使您没有太多编程经验,也不难识别模块中应包含的脚本块。例如,登录是一个显而易见的选择。
We’ve talked about some disadvantages of record/playback tools, but they’re appropriate in the right situation. You may be using a record/playback tool for a good reason: Maybe your legacy code already has a suite of automated tests created in that tool, your team has a lot of expertise in the tool, or your management wants you to use it for whatever reason. You can use recorded scripts as a starting point, then break the scripts into modules, replace hard-coded data with parameters where appropriate, and assemble tests using the modules as building blocks. Even if you don’t have much programming experience, it’s not hard to identify the blocks of script that should be in a module. Login, for example, is an obvious choice.
记录/回放也可能适用于遗留系统,这些遗留系统的设计方式使得单元测试变得困难,并且从头开始手工编写脚本测试的成本太高。可以在应用程序中构建记录和回放功能,甚至是遗留应用程序。通过正确的设计,并为记录的交互使用一些人类可读的格式,甚至可以在构建代码之前构建回放测试。
Record/playback may also be appropriate for legacy systems that are designed in such a way that makes unit testing difficult and hand-scripting tests from scratch too costly. It’s possible to build a record and playback capability into the application, even a legacy application. With the right design, and the use of some human-readable format for the recorded interaction, it’s even possible to build playback tests before the code is built.
一些敏捷团队从商业或开源测试工具中获得价值,而其他团队则更喜欢完全定制的方法。许多测试人员发现使用 Ruby 或 shell 等脚本语言编写简单的脚本来自动执行平凡但必要的任务、生成测试数据或驱动其他工具很有价值。Everyday Scripting with Ruby for Teams, Testers, and You等书籍给出了这种方法的路线图。如果您是一名没有强大编程背景的测试人员,我们鼓励您拿起一本书、查找在线教程或参加有关脚本语言的课程,看看编写有用的脚本是多么容易。
Some agile teams get value from commercial or open source test tools, while others prefer a completely customized approach. Many testers find value writing simple scripts in a scripting language such as Ruby, or a shell, to automate mundane but necessary tasks, generate test data, or drive other tools. Books such as Everyday Scripting with Ruby for Teams, Testers, and You give a roadmap for this approach. If you’re a tester without a strong programming background, we encourage you to pick up a book, find an online tutorial, or take a class on a scripting language, and see how easy it can be to write useful scripts.
在第 9 章“支持团队的面向业务测试的工具包”中查看面向业务测试的特定工具的更多示例。
See more examples of specific tools for business-facing tests in Chapter 9, “Toolkit for Business-Facing Tests that Support the Team.”
我们想告诉您的是,您可以使用许多不同的工具。看看你正试图解决的问题,然后作为一个团队来决定解决它的最简单和最有效的方法。每隔一段时间,退一步看看工具你正在使用。团队中的每个人都对他们满意吗?您是否因为没有合适的工具而遗漏了问题?预算时间探索新工具,看看它们是否可以填补空白或更换没有回报的工具。
What we’re trying to tell you is that you can use many different tools. Look at the problem you are trying to solve and decide as a team the easiest and most effective way to solve it. Every so often, step back and take a look at the tools you’re using. Is everyone on the team happy with them? Are you missing problems because you don’t have the right tools? Budget time to explore new tools and see if they might fill gaps or replace a tool that isn’t paying off.
如果您的团队是敏捷开发的新手,或者正在处理一个全新的项目,您可能会在早期迭代期间面临选择工具和设置测试环境的问题,同时您还可能处理高风险的故事。如果您仍在创建测试基础设施,就不要指望能够提供太多的商业价值。计划大量时间来评估工具、设置构建过程以及尝试不同的测试方法。
If your team is new to agile development, or working on a brand-new project, you might be faced with choosing tools and setting up test environments during the early iterations, when you also might be working on high-risk stories. Don’t expect to be able to deliver much business value if you’re still creating your test infrastructure. Plan in lots of time for evaluating tools, setting up build processes, and experimenting with different test approaches.
每个团队、每个项目和每个组织都有独特的情况和独特的自动化挑战。每个人都有自己的文化、历史、资源、业务压力、产品和经验。无论您的团队的情况如何,您都可以使用第 2 章中讨论的敏捷原则和价值观来帮助您找到解决方案。勇气、反馈、简单、沟通、持续改进和响应变化等概念不仅仅是敏捷的想法——它们是所有成功团队共有的品质。
Every team, every project, and every organization has a unique situation with unique automation challenges. Each has its own culture, history, resources, business pressures, products, and experience. No matter what your team’s situation, you can use the agile principles and values discussed in Chapter 2 to help you find solutions. Concepts such as courage, feedback, simplicity, communication, continuous improvement, and responding to change aren’t just agile ideas—they’re qualities that are common to all successful teams.
“做可能有效的最简单的事情”的敏捷格言适用于测试和代码。保持测试设计简单,范围最小,并使用最简单的工具来完成这项工作。
The agile maxim of “do the simplest thing that could possibly work” applies to tests as well as code. Keep the test design simple, keep the scope minimal, and use the simplest tool that will do the job.
简单是敏捷的核心价值,这是有充分理由的。最好的起点是您能想到的最简单的方法。然而,做最简单的事并不意味着做最容易的事。它涉及真正思考您现在需要什么,并采取一些小步骤来实现目标。通过保持简单,如果您确实做出了错误的选择,您就不会在意识到自己的方法错误之前偏离轨道太远。
Simplicity is a core agile value for a good reason. The best place to start is the simplest approach you can think of. However, doing the simplest thing doesn’t mean doing the easiest thing. It involves really thinking about what you need now and taking baby steps to get there. By keeping things simple, if you do make a bad choice, you won’t go too far off track before realizing the error of your ways.
很容易参与一项任务,并从基础知识中溜走,进入一些有趣的挑战中。在执行之前权衡每项自动化任务的投资回报率。自动化很有趣(当您克服入门的可怕部分时)。尝试一些困难的事情是很诱人的,因为你可以。与敏捷开发项目中测试的所有其他方面一样,跟上步伐的唯一方法是只做所需的最少工作。
It’s easy to get involved in a task and slip away from the basics into some intriguing challenge. Weigh the ROI of every automation task before you do it. Automation is fun (when you get past the scary part of getting started). It’s tempting to try something difficult just because you can. Like all other aspects of testing in an agile development project, the only way to keep up is to do only the minimum required.
使用您可以使用的最简单的工具。记住测试金字塔。如果面向客户的测试最容易在单元级别实现自动化,那就去做吧那里。Lisa 有时在 FitNesse 中编写测试用例,只是为了了解程序员可以像 JUnit 测试一样更快地自动化它们。相反,有时程序员将 FitNesse 用于 TDD 而不是 JUnit,因为他们编写的代码适合以其中一种 FitNesse 夹具格式进行测试。
Use the simplest tool you can get away with. Remember the test pyramid. If a customer-facing test can be most easily automated at the unit level, do it there. Lisa sometimes writes test cases in FitNesse, only to learn the programmers can automate them much faster as JUnit tests. Conversely, sometimes the programmers use FitNesse for TDD instead of JUnit, because the code they’re writing lends itself to testing in one of the FitNesse fixture formats.
短迭代使我们能够试验各种自动化方法、评估结果并根据需要尽快改变路线。致力于自动化工作,例如在内部开发测试框架或实施开源工具至少进行几次迭代。每次迭代后,查看哪些有效,哪些无效。想出克服问题的想法,并在下一次迭代中尝试这些想法。如果它不是正确的解决方案,请尝试其他方法进行几次迭代。不要陷入泥潭,你已经将如此多的资源投入到一个工具中,并且有如此多的测试使用它,以至于你觉得你无法切换工具。在许多开源和商业工具之间,加上程序员编写自己开发的测试工具的能力,没有理由满足于低于最佳的工具。
Short iterations allow us to experiment with various automation approaches, evaluate results, and change course as quickly as needed. Commit to an automation effort, such as developing a test framework in-house or implementing an open source tool for at least a couple of iterations. After each iteration, look at what’s working and what’s not working. Think of ideas to overcome problems, and try those in the next iteration. If it’s not the right solution, try something else for a few iterations. Don’t get sucked into a quagmire where you’ve put so many resources into a tool, and have so many tests that use it, that you feel you can’t switch tools. Between the many open source and commercial tools, plus programmers’ ability to write home-grown test tools, there’s no reason to settle for less than the optimum tool.
使用迭代对你有利。它们促进了循序渐进的方法。如果您的想法是一个失败的想法,您很快就会知道并有机会尝试不同的想法。不要害怕继续寻找,但如果您尝试的解决方案表现良好,也不要继续寻找完美的解决方案。
Use iterations to your advantage. They facilitate a step-wise approach. If your idea’s a dud, you’ll know quickly and have a chance to try a different one. Don’t be afraid to keep looking, but don’t keep looking for the perfect solution if one you try performs adequately.
没有自动化,敏捷开发就无法工作。幸运的是,我们在第 1 章中探讨的全团队方法意味着更广泛的技能和资源可用于找到和实施有用的自动化策略。作为一个团队解决问题意味着更有可能将代码设计为可测试性。程序员、测试人员和其他团队成员将协作实现测试自动化,为这项工作带来多种观点和技能组合。
Agile development can’t work without automation. Fortunately, the whole-team approach, which we explored in Chapter 1, means that a wider range of skills and resources are available to find and implement a useful automation strategy. Attacking the problem as a team means it’s more likely that code will be designed for testability. Programmers, testers, and other team members will collaborate to automate tests, bringing multiple viewpoints and skill sets to the effort.
整个团队的方法有助于克服恐惧障碍。自动化任务一开始可能会让人不知所措。知道还有其他具有不同技能和经验的人可以帮助我们,这给了我们勇气。能够寻求和获得帮助让我们相信我们可以通过自动化测试实现足够的覆盖率。
The whole-team approach helps overcome the fear barrier. Automation tasks can be overwhelming to start with. Knowing there are other people with different skills and experience to help gives us courage. Being able to ask for and receive help gives us confidence that we can achieve adequate coverage with our automated tests.
专门的面向技术的测试,例如安全或负载测试,可能需要从团队外部引入专家。一些公司拥有专家团队,可以作为产品团队的共享资源。即使在利用这些资源的同时,敏捷团队仍应负责确保完成所有类型的测试。他们还可能会惊讶地发现,如果团队成员采取创造性的方法,他们可能具备所需的技能。
Specialized technology-facing tests such as security or load testing might require bringing in experts from outside the team. Some companies have specialist teams that are available as shared resources to product teams. Even while taking advantage of these resources, agile teams should still take responsibility for making sure all types of testing are done. They may also be surprised to find that team members may have the skills needed if they take a creative approach.
第 11 章,“使用面向技术的测试来评价产品”,讨论了诸如此类的面向技术的测试以及处理它们的不同方法。
Chapter 11, “Critiquing the Product Using Technology-Facing Tests,” talks about technology-facing tests such as these and different approaches to handling them.
一些组织有独立的测试团队进行开发后测试。他们可能正在测试以确保软件与其他系统集成,或者进行其他专门测试,例如大规模性能测试。开发团队应该与这些其他团队密切合作,利用所有测试工作的反馈来改进代码设计并促进自动化。
Some organizations have independent test teams that do post-development testing. They may be testing to ensure the software integrates with other systems, or conducting other specialized testing such as large-scale performance testing. Development teams should work closely with these other teams, using feedback from all testing efforts to improve code design and facilitate automation.
解决问题和实施好的解决方案需要时间。我们必须帮助我们的管理层了解,如果没有足够的时间以正确的方式做事,我们的技术债务将会增加,我们的速度将会减慢。以“正确”的方式实施解决方案需要时间,但从长远来看会节省时间。考虑一下集思广益、解决方案、正式培训和在职学习所需的时间。
Solving problems and implementing good solutions takes time. We must help our management understand that without enough time to do things the right way, our technical debt will grow, and our velocity will slow. Implementing solutions the “right” way takes time up front but will save time in the long term. Consider the time it takes for brainstorming ideas, solutions, formal training, and for on-the-job learning.
可以理解,您组织的管理层对尽快产生结果很感兴趣。如果管理层不愿意给团队时间来实施自动化,请清楚地解释权衡取舍。在没有自动回归测试的情况下在短期内交付一些功能以确保它们继续工作将会付出巨大的代价。随着您的团队积累技术债务,您将无法满足业务价值管理需求。努力达成妥协。例如,缩小功能范围但保留基本价值,并使用自动化来交付和维护更好的产品。
Your organization’s management is understandably interested in producing results as quickly as possible. If management is reluctant to give the team time to implement automation, explain the trade-offs clearly. Delivering some features in the short term without automated regression tests to make sure they keep working will have a big cost down the line. As your team accumulates technical debt, you’ll be less able to deliver the business value management needs. Work toward a compromise. For example, cut the scope of a feature but keep the essential value, and use automation to deliver and maintain a better product.
我们总是有最后期限,我们总是感到时间紧迫。回到我们一贯的方式做事的诱惑,比如手动执行回归测试并希望获得最好的结果,即使我们知道这是行不通的,但这种诱惑总是存在的。永远没有足够的时间回去解决问题。在您的下一次计划会议期间,安排时间在您的自动化工作上取得有意义的进展。
We always have deadlines, and we always feel pressed for time. The temptation to just go back to doing things the way we always have, like executing regression tests manually and hoping for the best, is always there even though we know that doesn’t work. There is never enough time to go back and fix things. During your next planning meeting, budget time to make meaningful progress on your automation efforts.
让自己成功。以可持续的速度工作。边走边花时间重构,否则最终会搞得一团糟。作为测试人员,我们总是有许多不同的任务要做。如果您正在学习一种新工具或尝试自动化新测试,请不要同时处理多项任务。找一大块时间集中注意力。这很难,但不断切换齿轮更难。
Allow yourself to succeed. Work at a sustainable pace. Take the time to refactor as you go or you’ll end up with a mess eventually. As testers, we always have many different tasks to do. If you’re learning a new tool or trying to automate new tests, don’t multitask. Find a big block of time and focus. This is hard, but switching gears constantly is harder.
如果业务利益相关者对您的团队“完成它”不耐烦,请与他们一起分析问题。有什么风险?生产问题的成本是多少?发布 quick hack 有什么好处?它会增加多少技术债务?由自动化测试支持的可靠设计的长期投资回报是多少?每种方法将如何影响公司的盈利能力和客户满意度?无形成本如何,比如做低质量的工作对团队士气的影响?有时业务是正确的,但我们打赌您通常会发现前期投资得到回报。
If business stakeholders are impatient for your team to “just get it done,” analyze the problem with them. What are the risks? How much will a production problem cost? What are the benefits of releasing a quick hack? How much technical debt will it add? What’s the long-term return on investment of a solid design supported with automated tests? How will each approach affect company profitability and customer satisfaction? What about the intangible costs, such as the effect that doing poor-quality work has on team morale? Sometimes the business will be right, but we’re betting that you’ll usually find that up-front investment pays off.
每个人都以不同的方式学习,但是当您决定了如何使测试自动化时,请投入并开始进行。在Everyday Scripting with Ruby for Teams, Testers, and You [2007] 中,Brian Marick 建议通过编写程序来学习编程。犯错误!你遇到的问题越多,你就会学到越多。让某人与您配对将有助于加快学习速度,即使你们都不熟悉该工具或语言。
Everyone learns in different ways, but when you’ve decided how you’re going to automate a test, jump in and start doing it. In Everyday Scripting with Ruby for Teams, Testers, and You [2007], Brian Marick advises to learn to program by writing a program. Make mistakes! The more problems you have, the more you’ll learn. Getting someone to pair with you will help speed up learning, even if neither one of you is familiar with the tool or the language.
如果您没有人可以配对,请与“橡皮鸭”交谈:想象一下您正在向同事描述问题。解释的过程往往可以使问题的原因跳入视野。简单地大声朗读测试可以帮助您找到其中的弱点。
If you don’t have anyone to pair with, talk to the “rubber ducky”: Imagine you’re describing the problem to a coworker. The process of explaining can often make the cause of the problem jump into view. Simply reading a test aloud to yourself can help you find the weaknesses in it.
测试与生产代码一样有价值。事实上,没有测试支持的生产代码并不多。像对待所有代码一样对待测试。将其保存在与生产代码相同的源代码控制工具中。您应该始终能够识别与特定代码版本一起使用的测试脚本的版本。
Tests are just as valuable as production code. In fact, production code isn’t much without tests to support it. Treat your tests the same way you treat all code. Keep it in the same source code control tool as your production code. You should always be able to identify the versions of test scripts that go with a particular version of code.
结对、重构、简单设计、模块化和面向对象的设计、良好的标准、尽可能保持测试独立——所有优秀代码的品质也是优秀自动化测试的品质。敏捷开发有时会被不知情的人认为是混乱或松散的,而实际上它是高度纪律的。以最严格的纪律执行您的自动化任务,分小步进行,检查每一步是否成功。如果您正在编写自动化脚本,请先编写测试,就像任何敏捷程序员编写生产代码一样。不过,请记住简单性。除非有良好的投资回报率,否则不要编写包含大量逻辑的花哨测试脚本。这些测试需要测试并且维护成本更高。尽可能指定测试而不是编写代码,并始终使用尽可能简单的方法。
Pairing, refactoring, simple design, modular and object-oriented design, good standards, keeping tests as independent as possible—all of the qualities of good code are also qualities of good automated tests. Agile development is sometimes perceived by the uninformed to be chaotic or lax, when in fact it is highly disciplined. Undertake your automation tasks with the greatest discipline, proceeding in small steps, checking in each step that succeeds. If you’re programming automated scripts, write them test-first, just as any agile programmer would write production code. Keep simplicity in mind, though. Don’t write fancy test scripts with lots of logic unless there’s a good ROI. Those tests need testing and cost more to maintain. Specify tests when you can instead of coding them, and always go with the simplest approach possible.
我们怎么强调都不为过:测试自动化是一项团队工作。不同团队成员的不同经验、技能和观点可以一起工作,以提出最佳的自动化方法。创新——有创意。做适合你独特情况的事情,不管“常识”怎么说。
We can’t emphasize it enough: Test automation is a team effort. The varying experience, skills, and perspectives of different team members can work together to come up with the best approach to automation. Innovate—be creative. Do what works for your unique situation, no matter what the “common wisdom” says.
自动化工具只是其中的一块拼图。测试环境和测试数据是必不可少的组成部分。接下来我们来看测试数据。
Automation tools are just one piece of the puzzle. Test environments and test data are essential components. Let’s look at test data next.
无论我们使用什么工具来自动化测试,测试都需要数据来处理。理想情况下,他们需要与生产数据匹配的真实数据。然而,生产数据库通常包含大量数据,而且它们可能非常复杂。此外,数据库访问会以指数方式减慢测试速度。就像很多敏捷测试一样,它是一种平衡行为。
No matter what tool we use to automate tests, the tests need data to process. Ideally, they need realistic data that matches production data. However, production databases usually contain lots and lots of data, and they can be highly complex. Also, database access slows down tests exponentially. Like so much of agile testing, it’s a balancing act.
在我们撰写本书时,有几种很酷的工具可用于为各种输入字段和边界条件生成测试数据。Data Generator、databene benerator、testgen、Datatect 和 Turbo Data 等开源和商业工具可用于生成平面文件或直接向数据库表生成数据。这些工具可以生成种类繁多的不同类型的数据,例如姓名和地址。
As we write this book, there are several cool tools available to generate test data for all kinds of input fields and boundary conditions. Open source and commercial tools such as Data Generator, databene benerator, testgen, Datatect, and Turbo Data are available to generate flat files or generate data directly to database tables. These tools can generate huge varieties of different types of data, such as names and addresses.
使用 Ruby 或 Python 等脚本语言、Fit 或 FitNesse 等工具或 shell 脚本,使用自行开发的脚本生成测试数据也相当容易。
It’s also fairly easy to generate test data with a home-grown script, using a scripting language such as Ruby or Python, a tool such a Fit or FitNesse, or a shell script.
生成测试数据的脚本和工具不必很复杂。例如,PerlClip 只是将文本生成到 Windows 剪贴板中,以便可以将其粘贴到需要的地方。任何能够消除足够单调乏味、让您发现有关应用程序的潜在问题的解决方案都值得尝试。“可能可行的最简单的事情”绝对适用于为测试创建数据。您希望让您的测试尽可能简单和快速。
Scripts and tools to generate test data don’t have to be complex. For example, PerlClip simply generates text into the Windows clipboard so it can be pasted in where needed. Any solution that removes enough tedium to let you discover potential issues about the application is worth trying. “The simplest thing that could possibly work” definitely applies to creating data for tests. You want to keep your tests as simple and fast as possible.
您的首选测试应该尝试拥有可以完全在内存中运行的测试。他们仍然需要设置和拆除测试数据,但数据不会存储在数据库中。每个测试都是独立的,并且运行速度与任何测试一样快。数据库访问意味着 I/O 和磁盘本来就很慢。每次读取数据库都会减慢您的测试运行速度。如果您的目标是向团队提供快速反馈,那么您希望您的测试尽快运行。诸如内存数据库之类的假对象可以让测试做它需要做的事情,并且仍然提供即时反馈。
Your first choice for testing should try to have tests that can run completely in-memory. They will still need to set up and tear down test data, but the data won’t store in a database. Each test is independent and runs as quickly as any test could. Database access means I/O and disks are inherently slow. Every read to the database slows down your test run. If your goal is to give fast feedback to the team, then you want your tests to run as quickly as possible. A fake object such as an in-memory database lets the test do what it needs to do and still give instant feedback.
可以在xUnit 测试模式中找到各种类型的测试替身的综合解释和示例。请参阅参考书目以获取更多信息以及用于处理测试存根以及模拟和伪造对象的工具。
Comprehensive explanations and examples of various types of test doubles can be found in xUnit Test Patterns. See the bibliography for more information on that and tools for working with test stubs and with mock and fake objects.
因为很难获得测试自动化的牵引力,所以很容易说“好吧,我们有一些测试,它们确实需要几个小时才能运行,但总比没有测试好。” 数据库访问是导致测试缓慢的主要原因。继续采取小步骤尽可能地伪造数据库,并在不涉及数据库的情况下测试尽可能多的逻辑。如果这很困难,请重新评估您的系统架构,看看它是否可以更好地组织以进行测试。
Because it’s so difficult to get traction on test automation, it would be easy to say “OK, we’ve got some tests, and they do take hours to run, but it’s better than no tests.” Database access is a major contributor to slow tests. Keep taking small steps to fake the database where you can, and test as much logic as possible without involving the database. If this is difficult, reevaluate your system architecture and see if it can be organized better for testing.
DbFit、NdbUnit等工具可以简化数据库测试,实现测试驱动的数据库开发;请参阅参考书目以获取更多资源。
Tools such as DbFit and NdbUnit can simplify database testing and enable test-driven database development; see the bibliography for more resources.
如果您正在测试代码中的业务逻辑、算法或计算,您会对代码本身在给定特定输入时的行为感兴趣;你不关心数据从哪里来,只要它能准确地代表真实数据。如果是这种情况,请构建作为测试的一部分并且可以在内存中访问的测试数据,并让生产代码从中运行。模拟数据库访问和对象,并专注于测试的目的。测试不仅运行得更快,而且更容易编写和维护。
If you’re testing business logic, algorithms, or calculations in code, you’re interested in the behavior of the code itself given certain inputs; you don’t care where the data comes from as long as it accurately represents real data. If this is the case, build test data that is part of the test and can be accessed in memory, and let the production code operate from that. Simulate database access and objects, and focus on the purpose of the test. Not only will the tests run faster, but they’ll be easier to write and maintain.
为测试生成数据时,请尽可能使用反映测试意图的值。除非您完全确信每个测试都是独立的,否则请为每个测试生成唯一的测试值。例如,使用时间戳作为字段值的一部分。独特的数据是另一个安全网,可以防止测试与杂散数据相互感染。当您需要大量数据时,请尝试随机生成数据,但始终在测试结束时对其进行清理,以免其渗入下一个测试。我们认识到,有时您需要测试非常特定类型的数据。在这些情况下,随机生成的数据会破坏测试的目的。但是您可以使用足够的随机化来确保每个测试都有唯一的输入。
When generating data for a test, use values that reflect the intent of the test, where possible. Unless you’re completely confident that each test is independent, generate unique test values for each test. For example, use timestamps as part of the field values. Unique data is another safety net to keep tests from infecting each other with stray data. When you need large amounts of data, try generating the data randomly, but always clean it up at the end of the test so that it doesn’t bleed into the next test. We recognize that sometimes you need to test very specific types of data. In these cases, randomly generated data would defeat the purpose of the test. But you may be able to use enough randomization to ensure that each test has unique inputs.
如果被测系统对数据库的依赖很大,这个自然要测试。如果您正在测试的代码从数据库读取和/或写入数据库,那么在某些时候您需要对其进行测试,并且您可能至少需要一些回归测试来验证代码的数据库层。
If the system under test relies heavily on the database, this naturally has to be tested. If the code you’re testing reads from and/or writes to the database, at some point you need to test that, and you’ll probably want at least some regression tests that verify the database layer of code.
我们首选的方法是让每个测试都将它需要的数据添加到测试模式中,对数据进行操作,在数据库中验证结果,然后删除所有这些测试数据,以便可以重新运行测试而不影响其他后续测试。这支持了测试相互独立的想法。
Our preferred approach is to have every test add the data it needs to a test schema, operate on the data, verify the results in the database, and then delete all of that test data so the test can be rerun without impacting other subsequent tests. This supports the idea that tests are independent of each other.
另一种选择是拥有可以使用规范或种子数据库中的数据快速刷新的测试模式。这个想法是这个种子数据是真实生产数据的代表性样本。因为它是少量数据,所以每次需要运行一套回归测试时都可以快速重建它。
Another alternative is having test schemas that can quickly be refreshed with data from a canonical or seed database. The idea is that this seed data is a representative sample of real production data. Because it’s a small amount of data, it can be quickly rebuilt each time a suite of regression tests needs to be run.
这种方法还增加了运行测试所需的时间,但在回归套件开始时只需几分钟,而不是从每个单独的测试中抽出时间。这些测试仍然比不访问数据库的测试慢,但比必须费力地填充每个表中的每一列的测试快。
This approach also increases the time it takes to run tests, but it’s just a few minutes at the start of the regression suite rather than taking time out of each individual test. The tests will still be slower than tests that don’t access the database, but they’ll be faster than tests that have to laboriously populate every column in every table.
规范数据有很多用途。测试人员和程序员可以有自己的测试模式,可以随意刷新。他们可以进行手动和自动测试,而无需踩到其他任何人的测试。如果仔细选择数据,数据将比每个测试可以为自己构建的有限数据量更真实。
Canonical data has many uses. Testers and programmers can have their own test schema to refresh at will. They can conduct both manual and automated tests without stepping on anyone else’s testing. If the data is carefully chosen, the data will be more realistic than the limited amount of data each test can build for itself.
当然,与几乎所有事物一样,它也有缺点。规范数据可能很难跟上。当您需要新的测试场景时,您必须确定可用的生产数据,或者构建您需要的数据并将其添加到种子模式中。出于安全原因,您必须清理数据,掩盖真实人员的识别特征,使其无害。每次添加将表或列添加到生产数据库,您必须相应地更新您的测试模式。您可能必须每年前滚日期敏感数据,或进行其他大规模维护。您必须仔细选择哪些表应该刷新,哪些表不需要刷新,例如查找表。如果您必须添加数据以增加测试覆盖率,则刷新将花费更长的时间,从而增加触发它的构建过程的时间。正如我们一直强调的那样,您的自动化构建及时提供反馈非常重要,因此越来越长的数据库刷新会延长您的反馈周期。您还会失去与规范数据的测试独立性,因此如果一个测试失败,其他测试可能也会效仿。
Of course, as with practically everything, there’s a downside. Canonical data can be a pain to keep up. When you need new test scenarios, you have to identify production data that will work, or make up the data you need and add it to the seed schema. You have to scrub the data, mask real peoples’ identifying characteristics, making it innocuous for security reasons. Every time you add a table or column to the production database, you must update your test schemas accordingly. You might have to roll date-sensitive data forward every year, or do other large-scale maintenance. You have to carefully select which tables should be refreshed and which tables don’t need refreshing, such as lookup tables. If you have to add data to increase test coverage, the refresh will take longer to do, increasing the time of the build process that triggers it. As we’ve been emphasizing, it’s important that your automated builds provide feedback in a timely manner, so longer and longer database refreshes lengthen your feedback cycle. You also lose the test independence with canonical data, so if one test fails, others may follow suit.
Lisa 的团队成员运行他们的 GUI 测试套件,并且他们针对模式的一些功能回归测试每次运行都使用规范数据刷新。在极少数情况下,由于对种子数据的错误更新,测试会意外失败。决定是否向前“滚动”数据,例如,将 2008 年的行变成 2009 年的行,这会让人头疼。到目前为止,使用规范数据的投资回报率已经为团队所接受。Janet 目前的团队还使用种子数据对本地构建进行“中间层”测试。它适用于开发周期中的快速反馈。但是,测试环境和暂存环境使用生产数据的迁移副本。缺点是回归测试只能在构建的本地副本上运行。风险很低,因为他们实行“一次构建,多次部署”。
Lisa’s team members run their GUI test suites and some of their functional regression tests against schemas refreshed each run with canonical data. On rare occasions, tests fail unexpectedly because of an erroneous update to the seed data. Deciding whether to “roll” data forward, so that, for example, 2008’s rows become 2009’s rows, gets to be a headache. So far, the ROI on using canonical data has been acceptable for the team. Janet’s current team also uses seed data for its “middle layer” testing on local builds. It works well for fast feedback during the development cycle. However, the test environment and the staging environments use a migrated copy of production data. The downside is that the regression tests can only be run on local copies of the build. The risk is low because they practice “build once, deploy to many.”
测试尽可能像生产系统的能力对于大多数软件开发团队来说是必不可少的。但是,针对生产数据库的副本运行一套自动回归测试可能会运行得太慢而无法提供有用的反馈。此外,当您带来新副本以保持最新时,您不能真正依赖任何保持稳定的数据。通常,当您谈论功能或端到端测试时,生产数据库的克隆对于手动探索性测试最为有用。
The ability to test a system that is as much like production as possible is essential to most software development teams. However, running a suite of automated regression tests against a copy of a production database would probably run too slowly to be useful feedback. Besides, you couldn’t really depend on any data remaining stable as you bring over new copies to stay up-to-date. Generally, when you’re talking about functional or end-to-end testing, a clone of the production database is most useful for manual exploratory testing.
压力、性能和负载测试是自动化密集型的,需要一个能够密切模拟生产的环境,以便提供可以转化为实际操作的结果。可用性、安全性和可靠性是需要类似生产系统的测试的其他示例,尽管它们可能不涉及太多自动化。
Stress, performance, and load testing, which are automation–intensive, need an environment that closely simulates production in order to provide results that can translate to actual operations. Usability, security, and reliability are other examples of testing that needs a production-like system, although they may not involve much automation.
总有一个权衡;您的生产数据库可能很大,因此它既昂贵又缓慢,但它提供了最准确的可用测试数据。如果您的组织能够负担得起硬件和软件来存储用于测试目的的生产数据的多个副本,那么这是理想的选择。小公司可以具有资源限制,可能会限制可以存储在测试和暂存环境中的数据量。在这种情况下,您需要决定可以支持多少测试数据,并计划如何复制足够多的相关数据以使测试能够代表“现实生活”中使用的数据。或者您可以考虑对硬件进行投资,它的成本每天都在下降,以支持真正的生产风格环境。否则,您的测试结果可能会产生误导。正如我们在规范数据中提到的,您可能需要在使用数据之前清理数据。
There is always a trade-off; your production database might be huge, so it is expensive and slow, but it provides the most accurate test data available. If your organization can afford hardware and software to store multiple copies of production data for testing purposes, this is ideal. Small companies may have resource constraints that might limit the amount of data that can be stored in test and staging environments. In this case, you’ll need to decide how much test data you can support and plan how to copy enough relevant data to make the test representative of what’s used in “real life.” Or you may consider making the investment in hardware, which is getting less expensive every day, to support a real production style environment. Otherwise, your test results might be misleading. As we mentioned with the canonical data, you may need to scrub the data before using it.
数据迁移需要针对真实数据库进行测试。数据库升级脚本需要针对真实数据和数据库模式的最新已知版本运行。
Data migration needs to be tested against a real database. The database upgrade scripts need to be run against real data and against the last known release of the database schema.
如果您了解测试的目的,则可以更好地评估您的需求。例如,如果您不需要直接测试存储过程或 SQL 查询的速度,请考虑使用内存数据库等工具,它们的工作方式与真实数据库一样,但可以大大加快您的测试速度。当您需要模拟实际生产环境时,将整个生产数据库,如有必要。快速反馈是目标,因此请尽可能有效地平衡测试真实场景和发现缺陷。
If you understand the purpose of your tests, you can better evaluate your needs. For example, if you don’t need to test stored procedures or SQL queries directly for speed, consider tools such as in-memory databases, which work just like real databases but greatly speed up your tests. When you need to simulate the actual production environment, make a copy of the entire production database, if necessary. Quick feedback is the goal, so balance testing realistic scenarios with finding defects as efficiently as possible.
选择自动化工具的第一步是列出该工具需要为您做的所有事情。让我们考虑一下如何确定您的测试工具要求。
The first step in choosing an automation tool is to make a list of everything the tool needs to do for you. Let’s consider how you can decide on your test tool requirements.
在决定要应对的下一个自动化挑战之后,请考虑您的工具需求。您已经拥有哪些工具?如果您需要额外的,您可能想要与您现有的测试和开发基础架构很好地集成的东西。您是否需要一种工具来轻松集成到持续构建过程中?您现有的硬件是否支持您需要执行的自动化操作?设置第二个构建过程以运行功能测试可能需要额外的机器。
After deciding on the next automation challenge to tackle, think about your tool needs. What tools do you already have? If you need additional ones, you probably want something that integrates well with your existing testing and development infrastructure. Do you need a tool to easily integrate into the continuous build process? Will your existing hardware support the automation you need to do? Setting up a second build process to run functional tests may require additional machinery.
谁将使用您希望实施的测试工具?非程序员会编写测试用例吗?您的程序员是否也想要一个他们觉得舒服的工具?您是否有需要协作的分布式团队成员?
Who’s going to use the test tool you’re hoping to implement? Will non-programmers be writing test cases? Do your programmers want a tool they feel comfortable with as well? Do you have distributed team members who need to collaborate?
谁将自动化和维护测试?您团队中已有的技能很重要。您需要多少时间来安装工具并学习如何使用它?如果您的应用程序是用 Java 编写的,那么使用 Java 编写脚本的工具可能是最合适的。团队成员是否有使用特定工具的经验?是否有一个单独的测试团队在某种工具方面具有专业知识?如果您正在开始向敏捷开发过渡,并且您已经拥有一个测试自动化团队,那么利用他们的专业知识并继续使用他们知道的工具可能是有意义的。
Who will be automating and maintaining the tests? The skills already on your team are important. How much time do you have to get a tool installed and learn how to use it? If your application is written in Java, a tool that uses Java for scripting may be the most appropriate. Do team members have experience with particular tools? Is there a separate test team with expertise in a certain tool? If you’re starting the transition to agile development and you already have a team of test automators, it may make sense to leverage their expertise and keep using the tools they know.
您的工具要求取决于您的开发环境。如果您正在测试 Web 应用程序,而您选择的工具不支持 SSL 或 AJAX,您可能会遇到问题。并非每个测试工具都可以测试 Web 服务应用程序。嵌入式系统测试可能又需要不同的工具。第 12 章“测试象限总结”中的案例研究展示了一种使用 Ruby 测试嵌入式应用程序的方法。
Your tool requirements are dependent on your development environment. If you’re testing a web application, and the tool you choose doesn’t support SSL or AJAX, you may have a problem. Not every test tool can test web services applications. Embedded system testing can need different tools again. The case study in Chapter 12, “Summary of Testing Quadrants,” shows one way to use Ruby to test an embedded application.
当然,您正在自动化的测试类型是关键。安全测试可能需要高度专业化的工具。有许多现有的开源和性能的供应商工具,所以选择一个的工作并不是压倒性的。当您掌握一项挑战时,您将为下一项做好更充分的准备。Lisa 的团队花了几年时间在单元、集成和功能级别开发强大的回归测试套件。性能测试是他们的下一个痛点。从早期的自动化工作中吸取的教训帮助他们更好地确定测试工具的要求,例如报告结果的简便性、与现有框架的兼容性和脚本语言。
Of course, the type of testing you’re automating is key. Security testing probably needs highly specialized tools. There are many existing open source and vendor tools for performance, so the job of selecting one isn’t overwhelming. As you master one challenge, you’ll be better prepared for the next. It took Lisa’s team a couple of years to develop robust regression test suites at the unit, integration, and functional levels. Performance testing was their next area of pain. Lessons learned from the earlier automation efforts helped them do a better job of identifying requirements for a test tool, such as ease of reporting results, compatibility with existing frameworks, and scripting language.
编写一份包含所有工具要求的清单。其中一些可能会相互冲突或相互矛盾——“该工具需要足够简单,以便客户可以指定测试”或“测试应该易于自动化”。把它们写下来,这样你就可以找到合适的平衡点。然后开始做你的研究。
Write a checklist that captures all your tool requirements. Some of them might conflict with or contradict each other—“The tool needs to be easy enough so that customers can specify tests” or “The tests should be easy to automate.” Write them down so you can find the right balance. Then start doing your research.
您将需要不同的工具来满足不同的目的。实施新工具并学习使用它们的最佳方法可能很快就会让人不知所措。一次尝试一种工具,解决您最大的疼痛区域。给它足够的时间进行公平的审判并评估结果。如果它对您有用,请先掌握该工具,然后再继续处理下一个疼痛区域和下一个工具。多任务处理可能适用于某些情况,但新技术需要全神贯注。
You’re going to need different tools to serve different purposes. Implementing new tools and learning the best way to use them can get overwhelming pretty quickly. Try one tool at a time, addressing your greatest area of pain. Give it enough time for a fair trial and evaluate the results. If it’s working for you, master that tool before you go on to the next area of pain and the next tool. Multitasking might work for some situations, but new technology demands full attention.
当您确定了一种工具来满足特定需求时,请退后一步,看看您还需要什么。您的团队面临的下一个自动化挑战是什么?您刚刚为另一个目的选择的工具是否也能满足该需求,或者您是否需要开始新的选择过程?
When you’ve settled on a tool to address a particular need, take a step back and see what else you need. What’s the next automation challenge facing your team? Will the tool you just selected for another purpose work for that need, too, or do you need to start a new selection process?
如果您决定在自己的组织之外寻找工具,那么第一步就是找时间尝试一些工具。从一些基础研究开始:Internet 搜索、关于工具的文章和其他出版物以及邮件列表是获取想法的好地方。编制要考虑的工具列表。如果您的团队使用 wiki 或在线论坛工具,请发布有关工具的信息并开始讨论优缺点。
If you’ve decided to look outside your own organization for tools, the first step is to find time to try some out. Start with some basic research: Internet searches, articles and other publications about tools, and mailing lists are good places to get ideas. Compile a list of tools to consider. If your team uses a wiki or online forum tool, post information about tools and start a discussion about pros and cons.
评估工具的预算时间。一些团队每隔几个月就会进行一次“工程冲刺”或“重构迭代”,而不是交付业务优先考虑的故事,他们开始致力于减少技术债务、升级工具版本和试用新工具。如果您的团队还没有这些,请向您的管理层提出申请以获取它们。减少你的技术债务和建立良好的测试基础设施将提高您未来的速度和探索性测试的空闲时间。如果您从来没有时间让代码更易于维护或升级工具,技术债务就会拖累您的速度,直至停滞不前。
Budget time for evaluating tools. Some teams have an “engineering sprint” or “refactoring iteration” every few months where, rather than delivering stories prioritized by the business, they get to work on reducing technical debt, upgrading tool versions, and trying out new tools. If your team doesn’t have these yet, make a case to your management to get them. Reducing your technical debt and establishing a good testing infrastructure will improve your velocity in the future and free time for exploratory testing. If you never have time to make code easier to maintain or upgrade tools, technical debt will drag down your velocity until it comes to a halt.
当你有一个可能满足你要求的工具列表时,将可能性缩小到一两个,学习如何充分使用每个工具来尝试它,然后做一个尖峰:尝试一个简单但有代表性的场景,你可以扔掉. 根据要求评估结果。使用回顾来考虑利弊。
When you have a list of tools that may meet your requirements, narrow the possibilities down to one or two, learn how to use each one well enough to try it, and do a spike: Try a simple but representative scenario that you can throw away. Evaluate the results against the requirements. Use retrospectives to consider pros and cons.
您需要哪些资源来实施和使用该工具?该工具将对团队的生产力和速度产生什么影响?它会带来什么风险?从长远来看,它会让你做哪些你现在做不到的事情?
What resources do you need to implement and use the tool? What impact will the tool have on the team’s productivity and velocity? What risks does it pose? What will it allow you to do in the long term that you can’t do now?
选择你的最佳候选人并承诺尝试一段时间——足够长的时间来获得一些能力。确保尝试所有关键任务功能。例如,如果您的应用程序大量使用 Ajax,请确保您可以使用该工具进行自动化测试。在回顾中,看看哪些有效,哪些无效。对这样的想法持开放态度,即它可能不正确并且您必须将其丢弃并重新开始。不要觉得你必须继续使用这个工具,因为你已经在它上面投入了很多。
Pick your top candidate and commit to trying it for some period of time—long enough to get some competency with it. Make sure you try all your mission-critical functionality. For example, if your application uses a lot of Ajax, make sure you can automate tests using the tool. In retrospectives, look at what worked and what didn’t. Be open to the idea that it might not be right and that you have to throw it out and start over. Don’t feel you have to keep on with the tool because you have so much invested in it already.
我们都知道,没有可以解决所有自动化问题的“灵丹妙药”。降低你的期望,敞开心扉。创造性的解决方案既依赖于艺术,也依赖于科学。
We all know that there’s no “silver bullet” that can solve all your automation problems. Lower your expectations and open your mind. Creative solutions rely on art as much as science.
第 11 章,“使用面向技术的测试来评价产品”,展示了所选性能测试工具 JMeter 产生的结果示例。
Chapter 11, “Critiquing the Product Using Technology-Facing Tests, shows an example of the results produced by the performance test tool chosen, JMeter.
我们很幸运拥有范围广泛且不断增长的工具集可供选择:本土、开源、供应商工具,或组合any,都是可行的选择。有这么多选择,诀窍是知道在哪里寻找并找时间试用工具,看看它们是否符合您的要求。因为我们无法预测未来,所以可能很难判断每个潜在解决方案的投资回报率,但评估它们的迭代方法有助于找到正确的解决方案。
We’re lucky to have an already vast range and ever-growing set of tools to choose from: home-grown, open source, vendor tools, or a combination of any, are all viable alternatives. With so many choices, the trick is knowing where to look and finding time to try tools out to see if they fit your requirements. Because we can’t predict the future, it may be hard to judge the ROI of each potential solution, but an iterative approach to evaluating them helps get to the right one.
您的应用程序是否存在独特的测试挑战,例如嵌入式软件或与外部系统的集成?团队成员是否具备编写自己的测试框架或在现有开源工具之上构建测试框架的技能、时间和意愿?如果是这样,本土测试工具可能是最合适的。
Does your application present unique testing challenges, such as embedded software or integration with outside systems? Do team members have the skills, time, and inclination to write their own test framework or build one on top of an existing open source tool? If so, home-grown test tools may be the best fit.
敏捷开发的一个令人高兴的结果(或者可能是一个主要的成功因素)是许多程序员被“测试感染”。今天的开发工具和语言使自动化框架更容易构建。Ruby、Groovy、Rails 以及许多语言和框架都适合自动化。可以利用现有的开源工具,例如 Fit 和 HtmlUnit,并在它们之上构建自定义框架。
A happy result (or perhaps a major success factor) of agile development is that many programmers are “test infected.” Today’s development tools and languages make automation frameworks easier to build. Ruby, Groovy, Rails, and many languages and frameworks lend themselves to automation. Existing open source tools such as Fit and HtmlUnit can be leveraged, with custom frameworks built on top of them.
本土工具有很多优势。他们绝对是程序员友好的。如果您的团队正在编写自己的自动化框架,它们将根据您的开发和客户团队的需求进行精确定制,并与您现有的构建流程和其他基础设施集成——您可以使它们像以前一样易于执行和解释结果你需要。
Home-grown tools have many advantages. They’re definitely programmer-friendly. If your team is writing its own automation frameworks, they’ll be precisely customized to the needs of your development and customer teams, and integrated with your existing build process and other infrastructure—and you can make them as easy to execute and interpret results as you need.
当然,本土并不意味着免费。一个小团队可能没有带宽来编写和支持工具以及开发生产代码。具有独特需求的大型组织可能能够组建一个自动化专家团队,他们可以与测试人员、客户、程序员和其他人协作。如果您的需求如此独特以至于没有现有工具支持它们,那么自主开发可能是您唯一的选择。
Home-grown doesn’t mean free, of course. A small team may not have the bandwidth to write and support tools as well as develop production code. A large organization with unique requirements may be able to put together a team of automation specialists who can collaborate with testers, customers, programmers, and others. If your needs are so unique that no existing tool supports them, home-grown may be your only option.
许多编写自己工具的团队都慷慨地将它们提供给开源社区。因为这些工具是由受测试影响的程序员编写的,供应商工具无法满足他们的需求,所以它们通常是轻量级的,适合敏捷开发。其中许多工具都是测试先行开发的,您可以随源代码一起下载测试套件,使定制更容易、更安全。这些工具具有广泛的吸引力,具有对程序员和测试人员都有用的功能。这价格合适,但重要的是要记住购买价格只是任何工具成本的一小部分。
Many teams who wrote their own tools have generously made them available to the open source community. Because these tools were written by test-infected programmers whose needs weren’t met by vendor tools, they are usually lightweight and appropriate for agile development. Many of these tools are developed test-first, and you can download the test suite along with the source code, making customization easier and safer. These tools have a broad appeal, with features useful to both programmers and testers. The price is right, although it’s important to remember that purchase price is only a fraction of any tool’s cost.
并不是所有的开源工具都有很好的文档记录,培训可能是一个问题。然而,我们在许多会议和用户组会议上看到了关于使用这些工具的研讨会和教程。一些开源工具有优秀的用户手册,甚至有在线教程和预定课程。
Not all open source tools are well documented, and training can be an issue. However, we see seminars and tutorials on using these tools at many conferences and user group meetings. Some open source tools have excellent user manuals and even have online tutorials and scheduled classes available.
如果您正在考虑开源解决方案,请寻找活跃的开发人员和用户社区。是否有带宽很大的邮件列表?是否经常发布新功能?有没有办法报告错误,有人修复它们吗?其中一些工具比供应商工具具有更好的支持和更快的错误响应。为什么?编写它们的人也在使用它们,他们需要这些特性来测试他们自己的产品。
If you’re considering an open source solution, look for an active developer and user community. Is there a mailing list with lots of bandwidth? Are new features released often? Is there a way to report bugs, and does anyone fix them? Some of these tools have better support and faster response on bugs than vendor tools. Why? The people writing them are also using them, and they need those features to test their own products.
有关开源测试自动化工具的更多信息,请参阅第 9 章“支持团队的面向业务的测试工具包”。
See Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” for more about open source test automation tools.
商业工具被认为是一种安全的选择。很难批评某人选择了一个已经存在多年的知名工具。他们可能会附带手册、支持和培训。对于缺乏技术背景的测试人员或其他用户,最初的提升可能会更快。有些非常健壮且功能丰富。您的公司可能已经拥有一个,并且拥有一个知道如何使用它的专家团队。
Commercial tools are perceived as a safe bet. It’s hard to criticize someone for selecting a well-known tool that’s been around for years. They’re likely to come with manuals, support, and training. For testers or other users who lack a technical background, the initial ramp-up might be faster. Some are quite robust and feature-rich. Your company may already own one and have a team of specialists who know how to use it.
尽管它们与时俱进,但供应商工具历来对程序员不友好。他们倾向于使用程序员不想花时间学习的专有脚本语言。他们也往往是重量级的。测试脚本可能很脆弱,很容易被应用程序的小改动破坏,而且维护起来很昂贵。这些工具大多是录制脚本以供后续播放。从维护的角度来看,记录/回放脚本是出了名的昂贵。
Although they are changing with the times, vendor tools are historically programmer-unfriendly. They tend to use proprietary scripting languages that programmers don’t want to spend time learning. They also tend to be heavyweight. The test scripts may be brittle, easily broken by minor changes to the application, and expensive to maintain. Most of these tools are recording scripts for subsequent playback. Record/playback scripts are notoriously costly from a maintenance perspective.
Elisabeth Hendrickson [ 2008 ] 指出,诸如此类的专用工具可能需要测试自动化专家。诸如此类的孤岛可能会与敏捷团队对抗。我们需要能够促进先测试而不是后测试开发的工具。测试工具不应成为变革的阻碍。
Elisabeth Hendrickson [2008] points out that specialized tools such as these may create a need for test automation specialists. Silos such as these can work against agile teams. We need tools that facilitate test-first, rather than test-last development. Test tools shouldn’t stand in the way of change.
请参阅参考书目,了解 Elisabeth Hendrickson 对此主题的完整讨论。
See the bibliography for a full discussion by Elisabeth Hendrickson on this subject.
如果您的人员已经精通供应商工具,并且可能仅由开发团队的一个子集或与开发分开的团队使用该工具,那么供应商工具可能很有意义。Lisa 的前两个 XP 团队使用了供应商工具并取得了一定程度的成功。
If you have people already expert in a vendor tool, and a use for a tool that might be used only by a subset of the development team or a team separate from development, a vendor tool could make lots of sense. Lisa’s first two XP teams used a vendor tool with some degree of success.
在撰写本文时,更好的功能测试工具和 IDE 不断涌现。这些通过全局搜索/替换等功能促进了测试维护任务。Twist 是作为 Eclipse IDE 插件集合实现的工具示例,因此它可以利用强大的编辑和重构功能。
As of this writing, better functional test tools and IDEs are emerging. These facilitate test maintenance tasks with features such as global search/replace. Twist is an example of a tool implemented as a collection of plug-ins to the Eclipse IDE, so it can take advantage of powerful editing and refactoring features.
Elisabeth Hendrickson [ 2008 ] 列出了有效的敏捷测试自动化工具的一些特征。这些工具应该:
Elisabeth Hendrickson [2008] lists some characteristics of effective agile test automation tools. These tools should:
支持立即开始测试自动化工作,使用测试先行的方法
Support starting the test automation effort immediately, using a test-first approach
将测试的本质与实现细节分开
Separate the essence of the test from the implementation details
支持和鼓励测试自动化代码部分的良好编程实践
Support and encourage good programming practices for the code portion of the test automation
支持使用真实语言和真实 IDE 编写测试自动化代码
Support writing test automation code using real languages, with real IDEs
促进协作
Foster collaboration
第 III 部分,“使用敏捷测试象限”,特别是第 9 章,“支持团队的面向业务的测试工具包”,包含在敏捷项目中运行良好的测试自动化工具的示例。
Part III, “Using Agile Testing Quadrants,” and particularly Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” contain examples of test automation tools that work well on agile projects.
在评估工具时,请考虑必须以多快的速度解决您的首要自动化需求。您将从哪里获得支持以帮助实施它?团队需要什么培训,可以投入多少时间?您必须以多快的速度升级此工具?
While you’re evaluating tools, think about how quickly your top priority automation need must be addressed. Where will you get the support to help implement it? What training does the team need, and how much time will be available to devote to it? How quickly do you have to ramp up on this tool?
在查看工具时,请牢记所有这些限制。为了在短期内实现重要的自动化,您可能不得不接受一个不如您真正想要的强大的工具。请记住,没有什么是永恒的。您可以逐步构建自动化工作。在找到工具、技能和基础设施的正确组合之前,许多团队都经历过不成功的尝试。
Keep all of these constraints in mind when you’re looking at tools. You might have to settle for a less robust tool than you really want in order to get vital automation going in the short term. Remember that nothing’s permanent. You can build your automation effort step-by-step. Many teams experience unsuccessful attempts before finding the right combination of tools, skills, and infrastructure.
良好的面向对象设计并不是构建一套可维护的自动化测试的唯一关键。您还需要经常运行测试以获得您的团队需要的反馈。无论我们选择什么工具,都必须与我们的构建过程集成。易于解释的结果应该会自动出现。
Good object-oriented design isn’t the only key to building a suite of maintainable automated tests that pay off. You also need to run the tests often enough to get the feedback your team needs.. Whatever tools we choose must be integrated with our build process. Easy-to-interpret results should come to us automatically.
我们选择的工具必须在我们的平台上工作,并且必须与我们的其他工具共享和良好地发挥作用。我们必须不断调整它们以帮助解决当前的问题。构建每天都在中断吗?也许我们需要将我们的结果与实际的交通信号灯挂钩,以建立团队对其状态的认识。面向业务的测试是否失败?应该清楚地说明什么地方失败了。我们没有额外的时间来隔离问题。
The tools we choose have to work on our platforms, and must share and play well with our other tools. We have to continually tweak them to help with our current issues. Is the build breaking every day? Maybe we need to hook our results up to an actual traffic light to build team awareness of its status. Did a business-facing test fail? It should be plain exactly what failed, and where. We don’t have extra time to spend isolating problems.
这些担忧是图片的重要组成部分,但仍然只是图片的一部分。我们需要工具来帮助我们设计模拟生产的测试环境。我们需要一些方法来保持这些测试环境独立,不受程序员可能所做的更改的影响。
These concerns are an essential part of the picture, but still only part of the picture. We need tools that help us devise test environments that mimic production. We need ways to keep these test environments independent, unaffected by changes programmers might be making.
构建测试基础设施可能是一项巨大的投资,但这是我们的敏捷团队需要进行的一项工作,以便在测试自动化方面取得飞跃。需要确定和实施硬件、软件和工具。根据贵公司的资源,这可能是一个长期项目。集思广益应对短期的方法,同时计划如何将您真正需要的基础架构组合在一起,以最大限度地降低风险、提高速度并交付尽可能最好的产品。
Building test infrastructure can be a big investment, but it’s one our agile team needs to make to get a jump on test automation. Hardware, software, and tools need to be identified and implemented. Depending on your company’s resources, this might be a long-term project. Brainstorm ways to cope in the short term, while you plan how to put together the infrastructure you really need to minimize risk, maximize velocity, and deliver the best possible product.
假设我们需要一种方法来找到验证特定场景的测试,了解每个测试的作用,并了解它验证的应用程序的哪一部分。也许我们需要满足从每个需求到其代码和测试的可追溯性审计要求。自动化测试需要以与生产源代码相同的方式进行维护和控制。当您标记要发布的生产代码时,验证该功能的测试需要成为标记的一部分。
Let’s say we need a way to find the test that verifies a particular scenario, to understand what each test does, and to know what part of the application it verifies. Perhaps we need to satisfy an audit requirement for traceability from each requirement to its code and tests. Automated tests need to be maintained and controlled in the same way as production source code. When you tag your production code for release, the tests that verified that functionality need to be part of the tag.
这是一个派上用场的例子。我们刚刚在开发中的代码中发现了一个问题。它是一个新问题,还是它已经潜伏在代码中一段时间并且以某种方式被测试遗漏了?我们可以部署生产中的标签,尝试重现问题,并调查测试未发现问题的原因。Lisa 的团队最近遇到这样一种情况,即回归套件遗漏了一个错误,因为测试模式中缺少数据库约束。如果您不将测试代码版本与生产代码版本绑定,则很难查明此类问题。
Here’s an example where that comes in handy. We just found a problem in the code under development. Is it a new problem, or has it been lurking in the code for a while and somehow missed by the test? We can deploy the tag that’s in production, try to reproduce the problem, and investigate why the tests didn’t catch it. Lisa’s team recently had a situation where the regression suite missed a bug because a database constraint was missing in the test schema. That kind of problem is hard to pinpoint if you aren’t tying your test code versions to your production code versions.
许多工具都有自己的组织方式。例如,FitNesse 自带 wiki,具有层次结构,内置版本控制。在撰写本文时,FitNesse 开始提供对 Subversion 等源代码控制工具的支持。用其他测试工具(例如 Watir 和 Canoo WebTest)编写的脚本可以而且应该在与生产代码相同的源代码控制系统中维护,就像单元测试一样。
Many tools come with their own means of organization. For example, FitNesse comes with its own wiki, with a hierarchical organization, and built-in version control. As of this writing, FitNesse is starting to provide support for source code control tools such as Subversion. Scripts written in other test tools, such as Watir and Canoo WebTest, can and should be maintained within the same source code control system as production code, just as the unit tests are.
Test management helps your team answer questions such as the following:
哪些测试用例已经自动化?
Which test cases have been automated?
哪些还需要自动化?
Which still need automating?
哪些测试当前作为回归套件的一部分运行?
Which tests are currently running as part of a regression suite?
哪些测试涵盖哪些功能领域?
Which tests cover what functional areas?
XYZ 功能是如何设计的?
How is feature XYZ designed to work?
这个测试用例是谁写的?什么时候?最后是谁改的?
Who wrote this test case? When? Who changed it last?
这个测试作为回归套件的一部分多久了?
How long has this test been part of the regression suite?
因为我们编写测试的主要原因之一是指导开发,所以我们需要组织测试,以便团队中的每个人都能为每个故事找到合适的测试,并轻松确定测试涵盖的功能。因为我们使用测试作为文档,所以当存在关于系统应该如何运行的问题时,开发或客户团队中的任何人都可以快速找到特定测试是至关重要的。我们可能需要多种工具来满足不同的测试管理目标。
Because one of the primary reasons we write tests is to guide development, we need to organize tests so that everyone on the team can find the appropriate tests for each story and easily identify what functionality the tests cover. Because we use tests as documentation, it’s critical that anyone on either the development or customer team can find a particular test quickly when there’s a question about how the system should behave. We might need multiple tools to satisfy different test management goals.
很容易失去对测试脚本的控制。当测试失败时,您需要快速查明问题所在。您可能需要知道最近对测试脚本进行了哪些更改,这很容易通过源代码控制系统中可用的历史记录进行。您的客户团队还需要一种方法来跟踪项目进度,了解测试覆盖了多少代码,并可能自行运行测试。测试管理系统,就像测试本身一样,应该促进团队成员之间和不同团队之间的沟通和协作。
It’s easy to lose control of test scripts. When a test fails, you need to pinpoint the problem quickly. You may need to know what changes have been made recently to the test script, which is easy with the history available in a source code control system. Your customer team also needs a way to keep track of project progress, to understand how much of the code is covered with tests, and possibly to run tests themselves. Test management systems, like the tests themselves, should promote communication and collaboration among team members and between different teams.
参与交付软件的每个人都需要轻松访问测试和测试结果。管理测试的另一个方面是跟踪哪些测试来自先前的迭代并需要继续通过,而哪些测试正在推动当前迭代中的开发并且可能尚未通过。持续集成和构建过程运行测试以快速反馈进度并捕获回归失败。图 14-4显示了一个一目了然的测试结果报告示例。一次测试失败,并明确说明失败原因。
Everyone involved with delivering software needs easy access to tests and test results. Another aspect of managing tests is keeping track of what tests are from prior iterations and need to keep passing, versus tests that are driving development in the current iteration and may not be passing yet. A continuous integration and build process runs tests for quick feedback on progress and to catch regression failures. Figure 14-4 shows an example of a test result report that’s understandable at a glance. One test failed, and the cause of the failure is clearly stated.
如果您正在通过测试推动开发,并且其中一些测试尚未通过,那么构建应该不会失败。一些团队,例如 Lisa 的团队,只是将新测试排除在集成和构建过程之外,直到它们第一次通过。在那之后,他们总是需要通过。其他团队在构建过程本身中使用规则来忽略为覆盖当前正在开发的代码而编写的测试中的失败。
If you’re driving development with tests, and some of those tests aren’t passing yet, this shouldn’t fail a build. Some teams, such as Lisa’s, simply keep new tests out of the integration and build process until they pass for the first time. After that, they always need to pass. Other teams use rules in the build process itself to ignore failures from tests written to cover the code currently being developed.
与任何测试自动化工具一样,您可以使用自己开发的、开源的或商业的系统来解决您的测试管理问题。我们在评估测试工具一节中描述的相同标准可以应用于选择测试管理方法。
As with any test automation tool, you can solve your test management problems with home-grown, open source, or commercial systems. The same criteria we described in the section on evaluating test tools can be applied to selecting a test management approach.
测试管理是应用敏捷价值观和原则以及整个团队方法的另一个领域。从简单开始。分小步进行试验,直到找到源代码控制、存储库和构建管理的正确组合,使测试和生产代码保持同步。经常评估您的测试管理方法,并确保它适应所有不同的测试用户。确定有效的方法和缺失的方法,并计划任务甚至故事以尝试使用其他工具或流程来填补任何空白。请记住保持测试管理的轻量级和可维护性,以便每个人都会使用它。
Test management is yet another area where agile values and principles, together with the whole-team approach, applies. Start simple. Experiment in small steps until you find the right combination of source code control, repositories, and build management that keeps tests and production code in synch. Evaluate your test management approach often, and make sure it accommodates all of the different users of tests. Identify what’s working and what’s missing, and plan tasks or even stories to try another tool or process to fill any gaps. Remember to keep test management lightweight and maintainable so that everyone will use it.
不要害怕得到一些东西——任何东西——到位,即使它有些不足。成功最重要的因素是刚刚开始。许多(如果不是大多数的话)成功的团队都是从一个糟糕的流程开始的,但成功地将一个不充分的流程变成了对团队成功真正重要的东西,一次一个。与敏捷测试的许多方面一样,以微小的增量进行改进是成功的关键。
Don’t be afraid to get something—anything—in place, even if it’s somewhat deficient. The most important factor in success is to just get started. Many, if not most, successful teams have started with a poor process but managed to turn an inadequate process into something truly essential to the team’s success, one piece at a time. As with so many aspects of agile testing, improving in tiny increments is the key to success.
如果你不从某个地方开始,你将永远不会获得自动化的牵引力。召集整个团队并开始实验。没有适当级别的测试自动化,您的团队就无法做到最好。您需要正确的测试自动化来频繁地交付业务价值。一两年后,您会想知道为什么您认为测试自动化如此困难。
If you don’t start somewhere, you’ll never get traction on automation. Get the whole team together and start an experiment. Without the right level of test automation, your team can’t do its best work. You need the right test automation to deliver business value frequently. A year or two from now, you’ll wonder why you thought test automation was so hard.
在本章中,我们考虑了如何应用敏捷价值观、原则和实践来制定自动化战略。我们讨论了以下与自动化相关的主题:
In this chapter, we considered how to apply agile values, principles, and practices to develop an automation strategy. We discussed the following subjects related to automation:
使用敏捷测试象限来帮助确定您需要测试自动化的位置和时间。
Use the agile testing quadrants to help identify where you need test automation, and when you’ll need it.
测试自动化金字塔可以帮助您的团队在测试自动化方面做出正确的投资,从而获得最大回报。
The test automation pyramid can help your team make the right investments in test automation that will pay off the most.
应用敏捷价值观、原则和实践来帮助您的团队获得测试自动化的动力。
Apply agile values, principles, and practices to help your team get traction on test automation.
重复任务、持续集成和构建过程、单元测试、功能测试、负载测试和数据创建都是自动化的理想选择。
Repetitive tasks, continuous integration and build processes, unit tests, functional tests, load tests, and data creation are all good candidates for automation.
象限 3 测试,如可用性测试和探索性测试,可能会受益于一些自动化来设置测试场景和分析结果,但人类的本能、批判性思维和观察无法自动化。
Quadrant 3 tests such as usability testing and exploratory testing may benefit from some automation to set up test scenarios and analyze results, but human instincts, critical thinking, and observation can’t be automated.
一个简单的、全团队的方法,使用迭代反馈,并花足够的时间可以帮助你开始一个好的解决方案。
A simple, whole-team approach, using iterative feedback, and taking enough time can help you get started on a good solution.
在制定自动化策略时,从最痛苦的地方开始,考虑多层次的方法,并努力不断地重新审视和改进你的策略,而不是从一开始就达到完美。
When developing an automation strategy, start with the greatest area of pain, consider a multi-layered approach, and strive for continuously revisiting and improving your strategy rather than achieving perfection from the start.
在决定自动化什么时考虑风险和投资回报率。
Consider risk and ROI when deciding what to automate.
花时间边做边学;将敏捷编码实践应用于测试。
Take time to learn by doing; apply agile coding practices to tests.
决定是否可以简单地在内存中构建输入,或者是否需要数据库中的生产型数据。
Decide whether you can simply build inputs in-memory, or whether you need production-style data in a database.
提供测试数据,使测试独立、可重新运行并尽可能快。
Supply test data that will allow tests to be independent, rerunnable, and as fast as possible.
一次满足一种工具需求,确定您的需求,然后决定选择或构建哪种类型的工具来满足您的需求。
Take on one tool need at a time, identify your requirements, and decide what type of tool to choose or build that fits your needs.
使用良好的测试自动化开发实践,并花时间进行良好的测试设计。
Use good development practices for test automation, and take time for good test design.
自动化工具需要适应团队的开发基础设施。
Automated tools need to fit into the team’s development infrastructure.
版本控制自动化测试以及他们验证的生产代码。
Version-control automated tests along with the production code that they verify.
良好的测试管理确保测试可以提供系统和开发进度的有效文档。
Good test management ensures that tests can provide effective documentation of the system and of development progress.
立即开始测试自动化。
Get started on test automation today.
每当我们与敏捷开发相对较新的参与者进行教程、网络研讨会或问答环节时,我们总是会被问到诸如“在迭代的第一部分,在任何东西准备好测试之前,测试人员做什么?”这样的问题。或“用户验收测试在何处适合敏捷发布周期?” 在敏捷过程中,很容易阐述谁应该做什么以及何时做的理论,但我们发现根据自己的经验给出具体的例子是我们可以给敏捷新手的最好帮助。通过与许多不同的敏捷团队的交谈,我们了解到在适用于敏捷开发和测试的方面存在很多共性。
Whenever we do tutorials, webinars, or Q&A sessions with participants who are relatively new to agile development, we’re always asked questions such as “What do testers do during the first part of an iteration before anything’s ready to test?” or “Where does user acceptance testing fit into an agile release cycle?” It’s easy to expound on theories of who should do what and when, in an agile process, but we find giving concrete examples from our own experience is the best help we can give agile newbies. Through our talking to many different agile teams, we’ve learned that there’s a lot of commonality in what works well for agile development and testing.
在本书的这一部分,我们将在整个迭代过程中跟踪敏捷测试人员的生活。实际上,我们将探索的不仅仅是一次迭代。我们将从测试人员在发布或主题规划期间所做的事情开始,当团队查看它将为几个即将到来的迭代所做的工作时。我们将举例说明测试人员可以做些什么来帮助团队成员在开始迭代时迅速上手。我们将展示编码和测试如何成为交付软件的一个集成过程的一部分,并且我们将描述测试人员和程序员如何紧密和渐进地合作。我们将解释测试人员可以帮助他们的团队保持正轨和衡量进度的不同方式,包括衡量指标和处理缺陷的有用方法。我们将研究与测试相关的活动,这些活动涉及结束迭代并寻找改进下一个迭代的方法。
In this part of the book, we’ll follow an agile tester’s life throughout an iteration. Actually, we’ll explore more than just an iteration. We’ll start with what testers do during release or theme planning, when the team looks at the work it will do for several upcoming iterations. We’ll give examples of what testers can do to help the team members hit the ground running when they start the iteration. We’ll show how coding and testing are part of one integrated process of delivering software, and we’ll describe how testers and programmers work closely and incrementally. We’ll explain different ways that testers can help their teams stay on track and gauge progress, including useful approaches to metrics and handling defects. We’ll look at testing-related activities involved in wrapping up an iteration and finding ways to improve for the next one. Finally, we’ll examine a tester’s role in a successful release, including the end game, UAT, packaging, documentation, and training.
敏捷测试生活片段中描述的活动可以由团队中的任何人执行,而不仅仅是测试专家。在某些团队中,所有团队成员都可以并且确实执行任何任务,无论是开发、测试、数据库、基础设施还是其他任务。为简单起见,在本节中,我们假设我们正在跟踪主要角色是测试的人,因为他们帮助交付高质量的软件。
The activities described in this slice-of-life look at agile testing can be performed by anyone on the team, not only testing specialists. On some teams, all team members can, and do, perform any task, be it development, testing, database, infrastructure, or other tasks. For simplicity, in this section we’ll assume we’re following someone whose primary role is testing as they help to deliver high-quality software.
敏捷开发团队在每次迭代中完成故事并交付生产就绪软件,但提前规划大局或更大的功能块。一个主题、史诗或项目可能包含多个迭代。在本章中,我们将了解当团队花时间计划发布时测试人员会做什么。我们还考虑了一些方法来跟踪我们的开发是否按预期进行,或者是否需要更正路线。
Agile development teams complete stories and deliver production-ready software in every iteration but plan the big picture or a larger chunk of functionality in advance. A theme, epic, or project may encompass several iterations. In this chapter, we look at what testers do when their team takes time to plan their release. We also consider ways to track whether our development is proceeding as anticipated, or if course corrections are needed.
软件团队尝试敏捷开发的原因之一是因为他们知道长期计划行不通。大多数业务环境都是多变的,优先级每周甚至每天都在变化。敏捷开发应该避免“预先进行大设计”。我们中的大多数人都经历过制定计划,结果证明是在浪费精力。但是我们必须对我们的客户正在寻找什么以及我们如何交付它有一些了解,以便有一个良好的开端。幸运的是,敏捷方法可以使计划成为一种有用的方法,让我们在了解如何交付产品方面抢先一步。
One reason software teams try agile development is because they know long-range plans don’t work. Most business environments are volatile, and priorities change every week, or even every day. Agile development is supposed to avoid “big design up front.” Most of us have experienced making plans that turned out to be a waste of effort. But we have to have some understanding of what our customer is looking for and how we might deliver it in order to get off to a good start. Fortunately, an agile approach can make planning a useful way to give us a head start on knowing how we will deliver the product.
XP 团队可能每隔几个月就花一天时间进行发布计划。其他敏捷团队在准备开始一个主题、史诗或主要功能时会提前计划,我们将其视为一组相关的故事。他们努力理解主题或高水平发布。客户对我们应该提供什么的看法是什么?发布的目的是什么?大局是什么?它将为企业和客户带来什么价值?还涉及哪些其他团队或项目并需要协调?UAT什么时候开始?什么时候将代码发布到暂存区、生产区?如果我们走上正轨,我们需要知道哪些指标?这些一般问题在发布计划中得到解决。
XP teams may take a day every few months for release planning. Other agile teams do advance planning when getting ready to start on a theme, epic, or major feature, which we think of as a related group of stories. They work to understand the theme or release at a high level. What is the customer’s vision of what we should be delivering? What’s the purpose of the release? What’s the big picture? What value will it deliver to the business, to the customers? What other teams or projects are involved and require coordination? When will UAT take place? When will code be released to staging, to production? What metrics do we need to know if we’re on track? These general questions are addressed in release planning.
有些团队不会花太多时间做发布计划活动。优先级变化很快,即使在特定的功能主题内也是如此。没有人愿意预先做太多最终被浪费的工作。一些团队只查看前几个故事,以确保他们可以顺利开始。至少,团队希望了解足够多的信息,以便让他们的系统架构指向正确的方向,并开始处理前几个故事。
Some teams don’t spend much time doing release planning activities. Priorities change quickly, even within a particular theme of features. Nobody wants to do too much work up front that ends up being wasted. Some teams just look at the first couple of stories to make sure they can get a running start. At the very least, teams want to know enough to get their system architecture pointed in the right direction and get started on the first few stories.
这些计划会议并非旨在详细计划发布的每个迭代。而且我们知道我们无法准确预测每次迭代可以完成多少个故事。但是,我们确实知道我们的平均速度,因此我们可以大致了解发布的可能范围。团队讨论功能和故事,试图从 20,000 英尺的高度了解发布内容以及可能需要多少次迭代才能完成。我们都喜欢 Mike Cohn 在他的《敏捷估算和规划》一书中提出的发布规划方法[2005]。企业想要包含的故事是相对于彼此的大小,然后根据它们提供的价值对功能进行优先级排序。团队可以通过功能识别“薄片”以确定哪些故事绝对必须完成,哪些在范围内,哪些“最好有”可以推迟到以后。他们着眼于故事之间的依赖关系、相对风险和其他决定特性编码顺序的因素。故事编码的顺序与故事的大小一样重要,有时甚至更重要。团队希望在发布的第一次迭代中交付价值。
These planning meetings aren’t intended to plan every iteration of the release in detail. And we know we can’t predict exactly how many stories we can complete each iteration. However, we do have an idea of our average velocity, so we can get a general idea of the possible scope of the release. The team talks about the features and stories, trying to get a 20,000-foot view of what can go into the release and how many iterations it might take to complete. Both of us like Mike Cohn’s approach to release planning in his book Agile Estimating and Planning [2005]. Stories that the business wants to include are sized relative to each other, and then features are prioritized according to the value they deliver. The team may identify “thin slices” through the features to determine what stories absolutely have to be done, what’s in scope, what “nice-to-haves” could be put off until later. They look at dependencies between stories, relative risk, and other factors that determine the order in which features should be coded. The order in which stories are coded is as important, or sometimes more important, than the size of the stories. Teams want to deliver value the first iteration of the release.
发布计划让开发人员和客户有机会考虑计划的功能对更大系统的影响,澄清假设,并查看可能影响哪些故事首先完成的依赖关系。他们可能会考虑更高级别的测试,以及是否需要新的资源,例如测试环境和软件。
Release planning is a chance for the developers and customers to consider the impact of the planned features on the larger system, clarify assumptions, and look at dependencies that might affect what stories are done first. They may think about testing at a high level and whether new resources such as test environments and software will be needed.
让我们跟随我们的敏捷测试员完成发布计划活动,看看她如何通过她独特的视角和专注来贡献价值。
Let’s follow our agile tester through release planning activities and see how she contributes value through her unique perspective and focus.
敏捷团队估计每个故事的相对大小。一些团队随着他们的发展而调整规模,将估计推迟到他们实际完成故事的迭代。其他人甚至在发布计划之前召开会议来评估故事。一些开发人员和客户团队坐在一起,同时编写和估算故事的大小。规模调整的目的是让程序员让企业了解每个故事的成本,并帮助他们确定优先级并计划前几次迭代。合作多年的高效团队可能会采取不太正式的方法。对于新的敏捷团队,学习调整故事规模需要大量的实践和经验。正确调整每个故事的大小并不重要,但要足够接近以使客户了解故事的大小,以便他们可以根据更好的信息确定优先级。随着时间的推移,单个故事大小的变化将被平均化,我们发现一个主题或相关的一组故事所花费的时间与预期的差不多。
Agile teams estimate the relative size of each story. Some teams size as they go, delaying the estimation until the iteration where they’ll actually complete the story. Others have meetings to estimate stories even in advance of release planning. Some developer and customer teams sit together to write and estimate the size of stories all at one time. The goal of sizing is for the programmers to give the business an idea of the cost of each story and to help them prioritize and plan the first few iterations. High-functioning teams who’ve worked together for years may take a less formal approach. For new agile teams, learning to size stories takes a lot of practice and experience. It’s not important to get each story sized correctly but to be close enough to give customers some idea of how big the stories are so they can prioritize with better information. Over time, variations on individual story sizing will average out, and we find that a theme or related group of stories takes about the amount of time expected.
至于如何计算故事大小,不同的团队使用不同的技术,但同样,我们喜欢 Mike Cohn 确定故事大小的方法。我们根据故事点、理想的日子或简单的“小、中、大”来衡量。每个故事相对于其他故事的大小是重要因素。例如,向现有用户界面添加一个输入字段显然比从头开发一个全新的屏幕要小得多。
As far as how to calculate story size, different teams use different techniques, but again, we like Mike Cohn’s approach to determining story size. We size in story points, ideal days, or simply “small, medium, large.” The relative size of each story to others is the important factor. For example, adding an input field to an existing user interface is obviously much smaller than developing a brand new screen from scratch.
如果企业知道平均速度(团队每次迭代完成的故事点数)并且对每个想要完成的故事有初始规模估计,它就会知道实施给定主题可能需要多长时间。与任何其他开发方法一样,没有任何保证,因为估计就是这样。尽管如此,企业仍可以很好地计划以开展其日常活动。
If the business knows the average velocity (the number of story points the team completes each iteration) and has the initial size estimates of each story it wants to get done, it has an idea of how long it might take to implement a given theme. As with any other development methodology, there are no guarantees, because estimates are just that. Still, the business can plan well enough to conduct its usual activities.
我们的团队使用计划扑克(在 Mike Cohn 的《敏捷估算和规划》一书中有解释)来估算故事规模。在计划扑克中,每个团队成员都有一副牌。每张卡片上都有许多点。该过程从客户或产品所有者阅读故事并解释其目的和将提供的价值开始。他可能会列出一些满意的条件或高级测试用例。经过简短的讨论后,团队成员每人举起一张积分卡,代表他们认为从他们的角度来看这个故事有多“大”。他们讨论点值上的任何重大差异并再次估算,直到达成共识。图 15-1显示了团队成员正在谈论他们各自刚刚显示的点值。这需要一个快速的过程——关于细节的长时间讨论不会导致更准确的尺寸估计。
Our teams use planning poker (explained in Mike Cohn’s book Agile Estimating and Planning) to estimate story size. In planning poker, each team member has a deck of cards. Each card has a number of points on it. The process begins with the customer or product owner reading a story and explaining its purpose and the value it will deliver. He might list a few conditions of satisfaction or high-level test cases. After a brief discussion, team members each hold up a point card that represents how “big” they think the story is from their perspective. They discuss any big differences in point value and estimate again until they reach consensus. Figure 15-1 shows team members talking about the point values they each just displayed. This needs to be a quick process—long discussions about details don’t result in more accurate size estimates.
一些团队根据在规定时间内完成给定故事需要多少人来计算故事的相对大小。其他人估计一个人需要多少天才能完成它。使用一种对所有团队成员都有意义的测量方法,并提供估计之间的一致性。
Some teams figure the relative sizes of stories by how many people are needed to complete a given story in a set amount of time. Others estimate how many ideal days one person would need to finish it. Use a measurement that makes sense to all team members and one that provides consistency among estimates.
我们最喜欢的说法之一是,“没有测试就没有故事”。但是,我们遇到过一些团队,其中测试未包含在故事大小的估计中。在某些情况下,测试一项功能可能比编写代码花费更长的时间。
One of our favorite sayings is, “No story is done until it’s tested.” However, we’ve run across teams where testing wasn’t included in estimates of story size. In some cases, testing a piece of functionality might take longer than coding it.
根据我们的经验,测试人员通常与其他团队成员有不同的观点。他们通常对领域有广泛的了解,并且可以快速识别一个故事可能对系统其余部分产生的“连锁反应”。他们还倾向于考虑与可能需要完成的开发没有直接关系的活动,例如在新的或更改的界面上培训用户。
In our experience, testers usually have a different viewpoint than other team members. They often have a broad understanding of the domain and can quickly identify “ripple effects” that one story might have on the rest of the system. They also tend to think of activities not directly related to development that might need to be done, such as training users on a new or changed interface.
测试人员需要参与规模调整过程。一些团队认为只有程序员应该参与,但当测试人员积极参与时,他们可以帮助获得更准确的故事规模,这符合整个团队的最大利益。
Testers need to be part of the sizing process. Some teams think that only programmers should participate, but when testers are active participants, they can help to get a much more accurate story sizing, which is in the best interests of the whole team.
让我们假设我们有图 15-2中的故事来评估。
Let’s imagine we have the story in Figure 15-2 to size up.
After the product owner reads the story, the following discussion ensues:
产品负责人: “我们只是想为用户提供一些简单的方法来删除项目,但我们并没有考虑具体的实现方式。”
Product Owner: “We just want some easy way for users to delete items, but we don’t have a specific implementation in mind.”
测试人员: “他们应该能够一次删除多个项目吗?”
Tester: “Should they be able to delete several items at once?”
产品负责人: “哦,是的,让它尽可能简单。”
Product Owner: “Oh, yes, just make it as easy as possible.”
测试员: “如果他们不小心删除了他们想购买的商品怎么办?”
Tester: “What if they accidentally delete an item they wanted to buy?”
产品负责人: “有没有什么办法可以保存已删除的项目以供以后检索?”
Product Owner: “Is there some way the deleted items can be saved for later retrieval?”
程序员: “当然可以,但是你应该为此写一个新故事。现在,我们应该从基本的删除功能开始。”
Programmer: “Sure, but you should write a new story for that. For now, we should start with the basic delete functionality.”
测试人员: “上一个版本我们实现了愿望清单功能。您是否希望用户能够将商品从他们的购物篮移动到他们的愿望清单?那也将是一个新的故事。”
Tester: “Last release we implemented a wish list feature. Do you want users to be able to move items from their shopping basket to their wish list? That would be a new story also.”
产品负责人: “是的,那是我们想做的另外两个故事,这是肯定的。我会把它们写下来,我们也可以调整它们的大小。但如果必须的话,我们绝对可以将它们推迟到下一个版本。”
Product Owner: “Yes, those are two more stories we want to do, for sure. I’ll write those down, we can size them also. But we could definitely put them off until the next release, if we have to.”
测试人员: “此功能可能发生的最糟糕的事情是什么?”
Tester: “What’s the worst thing that could happen with this feature?”
产品负责人: “如果他们不知道如何删除,他们可能会放弃整个购物篮。它必须非常简单明了。”
Product Owner: “If they can’t figure out how to delete, they might just abandon their whole shopping basket. It has to be really easy and obvious.”
ScrumMaster 要求进行估算。该团队了解他们只是在调整删除项目的基本故事的大小,而不是为了对已删除的项目做其他事情。他们很快就点值达成一致。
The ScrumMaster calls for an estimate. The team understands they’re sizing only the basic story for deleting items, not for doing something else with the deleted items. They quickly agree on a point value.
我们再看一个故事。(见图15-3。)
Let’s look at another story. (See Figure 15-3.)
测试人员: “用户可以选择的运输速度是多少?”
Tester: “What are the shipping speeds the user can choose?”
产品负责人: “标准的 5 天、2 天和次日。”
Product Owner: “Standard 5-day, 2-day, and next-day.”
程序员: “我们或许应该从只提供一种速度开始,然后计算该成本。然后我们可以轻松实现其他两种速度。”
Programmer: “We should probably start by only offering one speed, and calculating that cost. Then we can easily implement the other two speeds.”
Product Owner: “就这样分解就好了。”
Product Owner: “It’s fine to break it up like that.”
测试人员: “我们会使用 BigExpressShipping 的 API 来根据重量和目的地计算费用吗?”
Tester: “Will we use BigExpressShipping’s API to calculate cost based on weight and destination?”
程序员: “那是最简单的。”
Programmer: “That would be the easiest.”
队员们举起他们的积分卡。测试人员和其中一名程序员举起 8;其他开发人员举起 5。
The team holds up their point cards. The tester and one of the programmers hold up an 8; the other developers hold up a 5.
ScrumMaster: “你们两个为什么选择8?”
ScrumMaster: “Why did you two choose 8?”
测试人员: “我们之前从未使用过 BigExpressShipping 的成本 API,我不确定这将如何影响我们的测试。我们必须找出如何访问他们的系统进行测试。”
Tester: “We’ve never used BigExpressShipping’s cost API before, and I’m not sure how that will impact our testing. We have to find out how to access their system for testing.”
其他 8 分的程序员: “我同意,我认为测试工作比这个故事的编码工作更紧张。”
Other Programmer with 8: “I agree, I think the testing effort is more intense than the coding effort for this story.”
团队同意将故事的大小分为八个要点。
The team agrees to size the story as eight points.
这个调整过程可能发生在计划会议之前,如果很久以前就对故事进行了调整或估算,团队可能希望确保他们对故事的大小感到满意。团队可能已经改变或可能更有经验。这些因素中的任何一个都可以使团队改变估计。
This sizing process may occur before the planning meeting, and if the stories were sized or estimated a long time ago, the team might want to make sure they feel comfortable with the story sizes. Teams may have changed or may be more experienced. Either of those factors can make a team change the estimates.
很多时候,一个故事会有很大的测试部分,而编码工作量很小。在其他时候,情况正好相反。考虑所有观点很重要。
There are many times when a story will have a large testing component, and the coding effort is small. At other times, the reverse will be true. It’s important to consider all perspectives.
发布计划会议的目的还在于了解团队将尝试在发布日期之前完成哪些故事。客户对故事进行优先级排序,但可能存在依赖关系,因此首先处理某些故事是有意义的,即使它们不是最高优先级。重要的是,团队了解并非所有故事都能在发布日期之前完成的可能性。敏捷的基本前提之一是交付可工作的软件,因此首先完成最高价值的故事非常重要,这样我们交付的软件才能满足客户的需求。
The purpose of the release planning meeting is also to get an idea of what stories the team will try to finish by the release date. The customers prioritize the stories, but there may be dependencies, so it makes sense to do certain stories first, even if they aren’t the highest priority. It is important that the team understands the possibility that not all of the stories will get completed by the release date. One of the basic premises of agile is to deliver working software, so it is important to have the highest-value stories completed first so that the software we do deliver meets the customer’s needs.
每个人的目标都是在每次迭代中交付真正的价值。测试人员可以帮助团队挑选出必须工作的核心功能。在第 8 章中,我们解释了“薄片”或“钢丝”概念,确定了通过功能首先进行编码和测试的路径,并在第一个关键路径起作用后添加更多功能。这个概念也适用于发布级别。故事的顺序很重要。Lisa 的团队有时会分解一个故事并抽出一个功能的核心部分在第一次迭代中完成。
Everyone’s goal is to deliver real value in each iteration. Testers can help the team pick out the core functionality that has to work. In Chapter 8, we explained the “thin slice” or “steel thread” concept, identifying one path through the functionality to code and test first, and adding more features after the first critical path works. This concept applies at the release level, too. The order of the stories is critical. Lisa’s team will sometimes break up a story and pull out a core part of a feature to do in the first iteration.
一些没有进行全面发布计划的团队确实会花时间查看故事并决定应该首先发布哪两三个。这样,他们在发布的第一次迭代中就提供了商业价值。
Some teams that don’t do full-blown release planning do take time to look at the stories and decide which two or three should be first. That way, they deliver business value in the very first iteration of the release.
让我们看一个例子。
Let’s look at an example.
如果我们的主题是为在线购物者提供选择运输选项的能力,然后根据重量、运输速度和目的地计算运输成本,那么完成简单的故事甚至故事的子集可能是个好主意,以便结账过程可以端到端地进行。首先只允许标准的 5 天送达、重量小于 10 磅的物品以及目的地在美国大陆。当用户可以获得该场景的运费并结账时,团队可以决定下一个优先级。它们可能包括重量级物品、更快的运输速度、运送到夏威夷和阿拉斯加以及运送到加拿大和墨西哥。
If our theme is providing the ability for an online shopper to choose shipping options and then calculate the shipping cost based on weight, shipping speed, and destination, it may be a good idea to complete simple stories or even subsets of stories so that the checkout process can proceed end-to-end. Start by only allowing standard 5-day shipping, items less than 10 pounds, and destinations in the continental United States. When the user can get the shipping cost for that scenario and check out, the team can decide the next priorities. They may include heavyweight items, faster shipping speeds, shipping to Hawaii and Alaska, and shipping to Canada and Mexico.
通过首先提供这个薄片,测试人员可以立即开始测试。程序员还测试了他们的设计和代码集成步骤,因此对整个功能完成后的工作方式有一个清晰的认识。
By providing this thin slice first, the testers have something to start testing immediately. The programmers have also tested their design and code integration steps and so have a solid idea of how things will work when the whole feature is complete.
团队了解大局或主题很重要。在我们的示例中,团队成员知道在美国大陆以外运输的故事将在稍后出现。这些知识可能会影响他们如何实施第一个故事。这并不意味着他们必须为每一种可能发生的情况做好计划,但如果他们知道他们需要更多的运输选项,他们可能会实施下拉列表而不是基本的文本字段。无需进行不必要的工作或返工。
It is important that the team understands the big picture or theme. In our example, team members know the stories for shipping outside the continental United States will come later. This knowledge may affect how they implement the first story. This doesn’t mean they have to plan for every eventuality, but if they know they need more shipping options, they may implement a drop-down list rather than a basic text field. No need to make more work or rework than necessary.
在发布计划期间,我们还考虑了故事的相对风险。如果某些故事有很多未知数,最好将它们包括在早期迭代中,这样如果故事“爆炸”并且花费的时间比估计的多得多,就有时间恢复。这同样适用于一个故事,如果没有完成或实施不当,将会产生代价高昂的负面影响。尽早安排将留出更多时间进行测试。
During release planning, we also consider the relative risk of the stories. If certain stories have many unknowns, it might be best to include them in an early iteration, so there’s time to recover if a story “blows up” and takes much more time than estimated. The same may apply to a story which, if not completed or implemented incorrectly, would have a costly negative impact. Scheduling it early will leave more time for testing.
如果需要新技术或软件,最好通过开发一个简单的故事来学习它,并为以后的迭代计划更困难的故事。这项新技术可能会也可能不会影响您的测试自动化。您可能需要更多时间来检查影响。如果这些功能都是全新的,并且团队需要更多时间来了解它们应该如何工作,那么计划在第一次迭代中以低于平均速度的速度进行。这样,您将有更多时间编写能够正确指导开发的测试。从测试角度和开发角度识别风险并确定哪种方法最有意义。这是让整个团队参与规划会议的重要原因之一。
If new technology or software is needed, it might be good to learn it by developing a straightforward story and plan more difficult ones for later iterations. This new technology may or may not affect your test automation. You may want more time to check out the impact. If the features are all brand new and the team needs more time to understand how they should work, plan to do less than your average velocity for the first iteration. That way, you’ll have more time to write tests that will correctly guide development. Identify risks and decide what approach makes the most sense from a testing perspective as well as a development perspective. This is one of the reasons it is important to include the whole team in the planning sessions.
从测试的角度来看故事是必不可少的。这是测试人员最有价值的地方。团队需要以可测试的小块进行开发,以帮助决定为哪个迭代暂定计划哪些故事。这里的关键是可测试的。许多新的敏捷团队认为小块意味着首先完成所有的数据库工作,或者所有的配置工作。可测试并不一定意味着它也需要 GUI。例如,计算运费的算法是一段独立的代码,可以独立于任何用户界面进行测试,但需要进行大量测试。这可能是第一次迭代的好故事。它可以作为独立代码进行测试,然后结合 UI 和系统的其他部分进行测试。
Looking at the stories from a testing viewpoint is essential. This is where testers add the most value. The team needs to develop in small, testable chunks in order to help decide what stories are tentatively planned for which iteration. The key here is testable. Many new agile teams think small chunks means doing all of the database work first, or all of the configuration stuff. Testable doesn’t necessarily mean it needs a GUI either. For example, the algorithm that calculates shipping cost is an independent piece of code that can be tested independently of any user interface but requires extensive testing. That might be a good story for the first iteration. It can be tested as free-standing code and then later tested in combination with the UI and other parts of the system.
测试人员可能会游说通过代码快速获得端到端的跟踪子弹,这样他们就可以构建一个自动化框架,然后充实它随着故事发展的进行。如果有故事提出了巨大的测试挑战,那么尽早完成这些故事可能会很好。例如,如果该版本包括实施新的第三方工具以从模板和动态数据创建文档,则有许多排列需要测试。如果团队不熟悉该工具,测试人员可以要求团队考虑在发布的第一次迭代中做那些故事。
The testers may lobby for getting an end-to-end tracer bullet through the code quickly, so they can build an automation framework, and then flesh it out as the story development proceeds. If there are stories that present a big testing challenge, it might be good to do those early on. For example, if the release includes implementing a new third-party tool to create documents from templates and dynamic data, there are many permutations to test. If the team is unfamiliar with the tool, the testers can ask the team to consider doing those stories in the first iteration of the release.
敏捷团队不断管理范围,以便在保持质量的同时满足业务期限。高价值的故事是第一要务。“锦上添花”的故事可能会被排除在发行版之外。
Agile teams continually manage scope in order to meet business deadlines while preserving quality. High-value stories are the first priority. Stories that are “nice-to-haves” might be elbowed out of the release.
Janet 与一个团队合作,该团队的客户错误地假设所有功能都会进入他们的版本,并且当他们确定优先级时,他们只是选择先完成哪些故事。当团队的其他成员意识到误解时,他们也实施了线上线下故事的想法。它有助于跟踪进度并使放在线下的故事非常明显。
Janet worked with a team whose customers were under the misplaced assumption that all of the features would get into their release and that when they were prioritizing, they were just picking which stories got done first. When the rest of the team realized the misunderstanding, they also implemented the idea of stories above and below the line. It helped to track progress as well as make the stories that were dropped below the line very visible.
许多领域围绕日历上的固定日期展开。零售企业在假期赚取大部分利润。一个 Internet 零售站点在 10 月 1 日之前实现所有新功能是明智的。临近购买高峰期的新特性是有风险的。丽莎公司的客户必须在一年中的特定时期完成政府要求的任务。如果某项功能今年发布为时已晚,则通常会推迟到明年发布,因为必须解决更紧迫的优先事项。监管变化有具体的时间表,组织对时间表没有选择。
Many domains revolve around fixed dates on the calendar. Retail businesses make most of their profit during the holiday season. An Internet retail site is smart to have all new features implemented by October 1. Implementing a new feature close to the peak buying period is risky. Lisa’s company’s customers must complete government-required tasks during certain periods of the year. When it’s too late for a feature to get released this year, it often gets put off for the next year, because more urgent priorities must be addressed. Regulatory changes have specific timelines, and organizations have no choice about the timeline.
团队很容易开始讨论一个复杂的故事,而忽视这些功能实际提供的价值。发布计划是时候开始询问有关如何使用这些功能以及它们将提供什么价值的示例和用例。在白板上绘制流程图或示例计算有助于确定核心功能。
It’s rather easy for a team to start discussing a complex story and lose sight of what value the features actually deliver. Release planning is the time to start asking for examples and use cases of how the features will be used, and what value they’ll provide. Drawing flowcharts or sample calculations on the whiteboard can help pinpoint the core functionality.
无法保证这些对给定版本中的内容的初始“猜测”会随着时间的推移而保持不变。这就是为什么客户需要了解他们的优先级,在每次迭代结束时采取检查点,并重新评估剩余故事的优先级。
There is no guarantee that these initial “guesstimates” at what will be in a given release will hold up over time. That is why customers needs to understand their priorities, take checkpoints at the end of every iteration, and reevaluate the priorities of remaining stories.
作为测试人员,我们的工作之一就是牢记大局。敏捷测试人员考虑每个故事如何影响整个系统或我们必须与之合作的其他系统。例如,如果玩具仓库对其库存软件进行了更改,并且新代码存在夸大库存商品数量的错误,则该网站可能会售出比可用的更多的热门新玩偶,让成千上万的孩子失望和他们的父母在圣诞节。当风险很高时,即使在发布计划期间,列出可能受主题或故事组影响的系统区域也是一项值得的练习。
One of our jobs as testers is to keep the big picture in mind. The agile tester thinks about how each story might affect the system as a whole, or other systems that ours has to work with. For example, if the toy warehouse makes a change to its inventory software, and the new code has a bug that overstates the number of items in stock, the website might sell more of the hot new doll than there are available, disappointing thousands of children and their parents at Christmas. When risk is high, listing areas of the system that might be affected for a theme or group of stories might be a worthwhile exercise even during release planning.
我们在第 8 章“支持团队的业务面临的测试”中讨论了“涟漪效应” 。
We talked about the “ripple effects” in Chapter 8, “Business Facing Tests that Support the Team.”
我们的系统与合作伙伴或供应商的系统之间的联系点始终值得考虑。如果我们没有正确地将其传达给通过 ftp 文件发送给我们的合作伙伴,即使是对 csv 或 xml 文件格式的微小更改也可能产生巨大影响。意味着需要在发布周期中尽早完成第三方更改的故事,以便让第三方进行必要的更改。
Contact points between our system and that of partners or vendors always merit consideration. Even a minor change to a csv or xml file format could have a huge impact if we don’t communicate it correctly to partners who ftp files to us. Stories that mean changes for third parties need to be done early enough in the release cycle to let the third parties make necessary changes.
图 15-5显示了一个涉及现有系统许多部分的新系统的简化图。可能需要不同的工具来测试集成。
Figure 15-5 shows a simplified diagram of a new system that touches many pieces of the existing system. Different tools might be needed to test the integrations.
使用过其他一些系统或了解需要在这些系统上进行哪些测试的测试人员可以提供有关新故事影响的宝贵见解。如果还没有探索影响,故事通常需要推迟到未来发布。现在是回顾之前结局不太好的版本的好时机。
Testers who have worked with some of the other systems or understand what testing needs to happen on those systems can offer valuable insight into the impact of a new story. Often, stories will need to be delayed until a future release if the impact has not been explored. This is a good time to recall previous releases that didn’t end so well.
在大型项目中与供应商工具、合作伙伴或其他承包商团队合作会使发布计划复杂化。如果您团队以外的任何人负责项目的某些部分,那么这就是您无法控制的一部分。如果您需要与其他人协调,包括系统可能的新用户,最好早点开始。
Working with vendor tools, partners, or other contractor teams on a big project complicates release planning. If anyone outside your team is responsible for some part of the project, that’s one piece that’s out of your control. If you need to coordinate with others, including possible new users of the system, it’s best to start early.
Lisa 的团队编写了多个接口,允许用户将数据上传到他们的系统。在每种情况下,他们都必须尽早将建议的文件格式提供给用户,以确保它适用于他们。其他项目涉及向合作伙伴或供应商发送数据。这些需要额外的计划来安排使用他们的测试系统进行测试,并获得他们对数据是否有效和格式是否正确的反馈。
Lisa’s team has written several interfaces to allow users to upload data to their systems. In each case, they had to get the proposed file format out to the users early to make sure it would work for them. Other projects involved sending data to partners or vendors. These required extra planning to arrange testing with their test systems and getting their feedback on whether data was valid and correctly formatted.
如果您使用第三方产品作为解决方案的一部分,您可能会假设它已经过测试,但这可能是一个糟糕的假设。您将需要预算额外的时间来结合供应商软件测试您的应用程序。如果其他公司的软件有问题,可能需要很长时间才能解决。Lisa 的团队使用第三方软件来完成文档创建等关键任务。如果主题包括修改或创建新文档,他们会计划额外的时间在需要时升级软件,以及额外的需要时间进行测试以防需要修复。如果可能,尽早将第三方软件引入项目,并开始端到端测试。您可以使用该界面的次数越多,您的生活就会越好。
If you’re using a third-party product as part of your solution, you might assume it has been tested, but that might be a poor assumption. You will need to budget extra time to test your application in conjunction with the vendor software. If there’s a problem in the other company’s software, it might take a long time to resolve. Lisa’s team uses third-party software for critical tasks such as document creation. If a theme includes modifying or creating new documents, they plan extra time to upgrade the software if needed, and extra time for testing in case fixes are needed. If possible, bring the third-party software into the project early, and start end-to-end testing. The more you can work with the interface, the better off you’ll be.
我们经常忘记直到为时已晚的其他第三方软件是我们自己的测试环境。有时,一个团队会合并新代码,以利用他们所选语言的新功能。例如,如果团队成员使用 AJAX 或 JavaScript,他们可能需要升级他们正在使用的软件开发工具包。这意味着团队还必须升级其生产运行时环境,因此请考虑到这一点并尽早进行测试。
Other third-party software that we often forget about until it’s too late is our own testing environments. Sometimes a team will incorporate new code that takes advantages of new features in their chosen language. For example, if team members are using AJAX or JavaScript, they may need to upgrade the software development kit they’re using. This means that a team will have to upgrade its production runtime environment as well, so take that into consideration and test early.
客户或合作伙伴可能对不在您自己团队范围内的发布感到担忧。Lisa 的团队曾在最后一刻被阻止发布一项新功能,因为合作伙伴没有时间与他们的法律顾问就变更达成一致。程序员必须快速设计出一种方法来关闭该功能,而无需进行大量的额外测试。有趣的是,不使用敏捷开发的合作伙伴有时会在截止日期前遇到困难。当您的团队在截止日期前完成时,他们可能毫无准备。
Clients or partners might have concerns about a release that isn’t within your own team’s scope. Lisa’s team was once prevented at the very last minute from releasing a new feature because a partner didn’t have time to okay the change with their legal advisors. The programmers had to quickly devise a way to turn the functionality off without requiring extensive additional testing. Interestingly, partners who aren’t using agile development sometimes have trouble meeting their own deadlines. They might be unprepared when your team meets the deadline.
如果您将与其他团队合作开发同一系统或相关系统的不同组件,请预留时间与他们协调。从每个团队中指定一名成员一起协调是个好主意。
If you’ll be working with other teams developing different components of the same system, or related systems, budget time to coordinate with them. It’s a good idea to designate a member from each team to coordinate together.
发布计划是确定团队中需要的额外角色、额外资源以及异常情况所需时间的时间。
Release planning is the time to identify extra roles you need on your team, additional resources, and time needed for out-of-the-ordinary circumstances.
我们不能期望在详细级别上计划发布中的迭代。我们可以了解主题的钢丝线,确定故事的优先级,并猜测哪些故事将出现在哪个迭代中。详细的测试计划需要等待迭代计划。尽管如此,我们仍需要考虑高层次的测试,并尝试为其预留足够的时间。我们甚至可能会在发布计划会议之外花时间来制定我们的发布测试策略。在第 8 章支持团队的面向业务的测试中,我们提到了敏捷测试的危险之一:“忘记大局”。测试计划将帮助您解决这个问题。
We can’t expect to plan the iterations in a release at a detailed level. We can get an idea of the theme’s steel threads, prioritize stories, and make a guess at what stories will be in which iteration. Detailed test planning needs to wait for iteration planning. Still, we need to think about testing at a high level, and try to budget enough time for it. We might even take time separately from the release planning meeting to strategize our testing for the release. In Chapter 8, Business-Facing Tests that Support the Team, we mentioned one of the perils of agile testing: “forgetting the big picture.” Test Planning will help you with that problem.
第 8 章,“支持团队的面向业务的测试”,解释了如何识别故事或主题中的钢线或薄片。
Chapter 8, “Business-Facing Tests that Support the Team,” explains how to identify steel threads or thin slices in a story or theme.
在发布计划期间,了解每个故事或高级用户验收测试用例的业务满意度条件很有帮助。当故事需要澄清时,敏捷测试人员会要求提供示例。在此阶段,示例将是高级示例,仅涵盖基础知识,但足以确定故事的大小和优先级。绘制流程图或在白板上写下计算结果并进行讨论有助于我们确定项目特定的测试问题。
During release planning, it’s helpful to know the business conditions of satisfaction for each story or high-level user acceptance test case. When stories need clarification, agile testers ask for examples. At this stage, examples will be high-level, covering just the basics, but enough to be able to size and prioritize the story. Drawing flowcharts or writing calculations on the whiteboard and discussing them helps us identify project-specific testing issues.
至少,团队需要了解计划首先执行的最高优先级故事。轻量级计划可能只涉及查看那些核心故事,并了解需要更多时间来定义额外的测试。
At a minimum, the team needs to understand the top-priority stories that are scheduled to be performed first. Lightweight planning might involve only looking at those core stories with the understanding that more time will be needed for defining additional tests.
当我们了解哪些故事可能会包含在发布中时,我们可以开始考虑测试的范围。做出了哪些可能影响测试的假设?使用第三方软件,例如使用航运公司的航运计算API的例子,影响测试计划。此版本中是否存在任何会影响测试的异常风险?如果我们有实现批处理作业的故事,并且我们以前从未在系统中进行过任何批处理,那么可能会有影响测试的新框架。我们需要预算时间来学习它们。
As we get a sense of which stories will probably be included in the release, we can start thinking about the scope of the testing. What assumptions have been made that might affect testing? Use of third-party software, such as the example of using a shipping company’s shipping calculation API, affects test planning. Are there any unusual risks in this release that will impact testing? If we have stories to implement batch jobs, and we’ve never had any batch processing in the system before, there are probably new frameworks that impact testing. We need to budget time to learn them.
在发布计划中,我们讨论发布的目的、范围内的内容以及我们所做的假设。我们进行一些快速的风险分析并计划我们的测试方法来解决这些风险。我们考虑自动化以及我们对测试环境和测试数据的需求。我们当然想确定里程碑和可交付成果。嗯,这听起来像是。. . 一个测试计划!
In release planning, we talk about the purpose of the release, what’s in scope, and what assumptions we’re making. We do some quick risk analysis and plan our test approach to address those risks. We consider automation and what we need for test environments and test data. We certainly want to identify milestones and deliverables. Hmmm, this is starting to sound like . . . a test plan!
如果像我们一样,您花时间在传统的瀑布环境中工作,您可能会浪费时间编写没有人阅读也没有人费心维护的庞大测试计划。在敏捷开发中,我们希望我们的测试计划能够满足我们的需求。出于合规性原因,您的客户可能需要每个版本的测试计划。即使它不是必需的可交付成果,它也很有用。保持简洁和轻便。它只需要在此版本中为您服务。解决特定于此版本或项目的测试问题。包括您的风险分析并确定假设。概述您的客户已确定的关键成功因素。考虑人们需要了解与测试相关的内容,并删除任何无关的内容。
If, like ourselves, you spent time working in a traditional waterfall environment, you might have wasted time writing bulky test plans that nobody read and nobody bothered to maintain. In agile development, we want our test plans to serve our needs. Your customer might require a test plan for each release for compliance reasons. Even if it’s not a required deliverable, it can be useful. Keep it concise and lightweight. It only has to serve your purposes during this release. Address the testing issues that are specific to this release or project. Include your risk analysis and identify assumptions. Outline the critical success factors that your customer has identified. Think about what people need to know related to testing, and remove anything extraneous.
有关测试计划和测试策略的更多信息,请参阅第 5 章“转换典型过程”。
See Chapter 5, “Transitioning Typical Processes,” for more about test plans and test strategies.
即使您没有创建正式的测试计划,也请确保您已记下发布中涉及的所有这些不同的测试因素。在每次迭代计划会议期间,您都希望将它们牢记在心。测试计划的最大好处是计划本身。它允许您考虑和解决诸如测试数据要求、基础设施甚至需要什么测试结果等问题。测试计划是一种风险缓解策略。让我们考虑其中的一些问题。
Even if you don’t create a formal test plan, be sure you have made note of all these different testing factors involved in the release. You’ll want to keep them in mind during every iteration planning session. The biggest benefit in test planning is the planning itself. It allows you to consider and address issues such as test data requirements, infrastructure, or even what test results are required. Test planning is a risk mitigation strategy. Let’s consider some of these issues.
在第 III 部分中,我们介绍了测试的四个象限,并讨论了您在项目中可以进行的所有不同类型的测试。发布计划是考虑这些不同需求的好时机。您是否需要计划引入负载测试工具,或者是否需要构建某种专业测试工具?
In Part III, we covered the four quadrants of testing and talked about all of the different types of testing you can do during your project. Release planning is a good time to consider these different needs. Do you need to plan to bring in a load test tool, or will there be the need to build some kind of specialty test harness?
可能你的下一个版本只是你上一个版本的扩展,你将继续创建你的例子,自动化你的故事测试,并像你一直在做的那样做其余的测试。你是幸运者之一。对于那些开始一个全新项目且之前没有任何流程的人来说,现在是时候考虑您需要进行哪些测试了。我们并不是说您必须决定如何测试每个故事,而是要着眼大局并考虑象限。您是否需要计划一个特殊的 UAT,或者迭代演示是否足够?尽早提出这些问题很重要,这样团队才能为它们做好计划。
It could be that your next release is just an extension of your last, and you will just carry on creating your examples, automating your story tests, and doing the rest of the testing as you’ve been doing. You are one of the lucky ones. For those of you who are starting a brand new project with no previous processes in place, now is the time to consider what testing you will need. We don’t mean you have to decide how to test each story, but look at the big picture and think about the quadrants. Will you need to plan for a special UAT, or will the iteration demos be enough? It is important to raise these issues early so the team can plan for them.
在进行测试计划时,您需要考虑您的基础设施。基础设施可能意味着您的持续集成设置、测试环境和测试数据库。这可能意味着您如何推广您的构建到您的测试环境。这可能意味着你的测试实验室,如果你有一个,或者有一个单独的服务器来运行你所有的自动化测试。这些通常是需要一些准备时间才能到位的基础设施。这是制定计划的时候了。
While you are doing your test planning, you need to consider your infrastructure. Infrastructure can mean your continuous integration setup, test environments, and test database. It can mean how you promote your builds to your test environments. It might mean your test lab, if you have one, or having a separate server to run all your automation tests. These are generally pieces of infrastructure that need some lead time to get in place. This is the time to make a plan.
无论您的需求是什么,请确保您了解它们并且可以针对您的需求进行规划。如果您没有合适的基础架构,那么您将浪费时间尝试将其组合在一起,并在迭代中期造成瓶颈。
Whatever your needs are, make sure you understand them and can plan for what you need. If you don’t have the right infrastructure, then you will waste time trying to get it together and cause a bottleneck in mid-iteration.
当我们查看下一个版本中的功能类型时,我们可能会发现需要一个全新的测试环境。考虑您可能还需要的专门测试环境。您需要更多工具吗?您是否需要扩展您的测试实验室以便您可以使用不同的浏览器和操作系统进行测试?现在是考虑所有测试注意事项的时候了。
As we look at the types of features in the next release, we might see the need for a whole new test environment. Think about specialized test environments you may need as well. Will you need more tools? Do you need to expand your test lab so that you can test with different browsers and operating systems? This is the time to think about all testing considerations.
如果您正在计划您的第一个版本,测试环境是一个关键的考虑因素。您可能需要一个故事或迭代来设置您需要的基础设施。我们已经启动了不止一个项目,我们唯一可以测试的地方就是开发环境。我们发现效果不是很好,因为环境永远不够稳定,无法进行有效测试。
If you’re planning your first release, test environments are a key consideration. You might need a story or iteration just to set up the infrastructure you need. We’ve started more than one project where the only place we could test was the development environment. We found that doesn’t work very well, because the environment is never stable enough for effective testing.
就像程序员有自己的沙箱来工作和测试一样,如果每个测试人员都具有相同的可用性和控制权,它就会运作良好。我们认识到并非所有应用程序都适用于此,但至少,您需要知道您正在测试的是什么版本。您还需要其他人不会绕过他们的测试的测试数据。如果您没有自己控制的测试沙箱,请花时间计划您需要为测试环境建立的内容。与您的团队集思广益,商讨如何获得必要的硬件和软件。这可能需要时间,因此制定一个 B 计划,在等待您需要的基础设施的同时完成一些事情。
Just as programmers have their own sandboxes to work and test in, it works well if each tester has that same availability and control. We recognize that not all applications lend themselves to this, but at the very least, you need to know what build you’re testing. You also need test data that others will not walk over with their tests. If you don’t have a testing sandbox that’s under your own control, take time to plan what you need to establish for your test environments. Brainstorm with your team about how you can obtain the necessary hardware and software. It might take time, so develop a Plan B for getting something done while waiting for the infrastructure you need.
如果您在大型系统上工作,您可能不得不与其他团队一起排队,以便在测试或暂存环境中获得时间,其中包括您必须使用的所有各种软件。此暂存环境应尽可能模仿您的生产系统。如果您的组织没有专人负责创建环境,您的团队可能需要额外的角色来专门获取您需要的测试环境。这些角色也可能涉及与其他团队合作。发布计划是时候考虑所有这些测试基础架构需求了。
If you’re working on a large system, you may have to queue up along with other teams to get time on a test or staging environment that includes all of the various pieces of software with which yours must work. This staging environment should mimic your production system as much as possible. If your organization doesn’t have someone responsible for creating environments, your team might need extra roles dedicated to obtaining the test environments you need. These roles might involve working with other teams as well. Release planning is the time to consider all of these testing infrastructure requirements.
发布或主题规划也是考虑项目期间可能需要哪些测试数据的好时机。
Release or theme planning is also a good time to think about what test data you might need during the project.
使用与真实数据非常相似的测试数据通常是一种很好的做法。规划您需要的数据。我们有机会在多个组织中使用生产数据的副本。真实数据为探索性测试的不同场景提供了良好的基础。生产数据在用于测试之前可能需要“清理”,以删除任何敏感信息,例如身份或银行帐号。需要更改数据以隐藏原始值但保持有效,以免违反数据库限制。由于数据库专家将生产数据移植到测试环境需要时间,因此请确保将它们包含在您的计划中。
Using test data that closely resembles real data is generally a good practice. Plan for the data you need. We’ve had the opportunity in several organizations to use a copy of production data. Real data provides a good base for different scenarios for exploratory testing. Production data may need to be “scrubbed” before it’s used for testing in order to remove any sensitive information such as identification or bank account numbers. The data needs to be altered to hide the original values but remain valid so that it doesn’t violate database restrictions. Because it takes time for database experts to port production data to a test environment, be sure they’re included in your planning.
争取客户的支持以获取有意义的测试数据。如果您正在处理涉及将文件发送给第三方供应商的故事,您的业务专家可以找出供应商希望文件中包含哪些数据。Lisa 的团队开发了允许退休计划经纪人向其客户提供共同基金组合的功能。他们要求产品负责人提供投资组合样本,包括每个投资组合的名称、描述和一组基金。这有助于他们使用真实数据进行测试。
Enlist your customers’ support in obtaining meaningful test data. If you’re working on a story that involves sending a file to a third-party vendor, your business expert can find out what data the vendor expects in the file. Lisa’s team developed features to allow retirement plan brokers to offer their customers portfolios of mutual funds. They asked the product owner to provide samples of portfolios, including a name, description, and set of funds for each. This helped them test with realistic data.
随着时间的推移,测试数据往往会变得陈旧和过时。较旧的数据,即使来自生产,也可能不再准确反映当前的生产数据。使用不再有效的数据进行的“通过”测试会给人一种误导性的自信感。持续审查您的测试数据需求。根据需要使用新方法刷新数据或创建数据。
Test data tends to get stale and out of date over time. Older data, even if it came from production, may no longer accurately reflect current production data. A “passing” test using data that’s no longer valid gives a misleading sense of confidence. Continually review your test data needs. Refresh data or create it using a new approach, as needed.
测试数据要求因测试类型而异。回归测试通常可以创建自己的数据或针对可以快速刷新到已知状态的小型代表性数据集运行。探索性测试可能需要生产类型数据的完整副本。
Test data requirements vary according to the type of testing. Regression tests can usually create their own data or run against a small representational set of data that can be refreshed to a known state quickly. Exploratory testing may need a complete replica of production type data.
第 14 章,“敏捷测试自动化策略”,探讨了获取测试数据的不同方法
Chapter 14, “An Agile Test Automation Strategy,” explores different approaches to obtaining test data
不同的团队对测试结果报告有不同的要求。考虑一下您将如何在游戏的这个阶段报告测试结果,以便在进行实际报告时可以有效地这样做。您的组织可能有审计合规性要求,或者您的客户可能只是想知道您是如何测试的。了解您的需求,以便您可以选择适合您团队的方法。
Different teams have different requirements for test result reporting. Think about how you are going to report test results at this stage of the game so that you can do so effectively when the time comes to do the actual reporting. Your organization may have audit compliance requirements, or maybe your customer just wants to know how you tested. Understand your needs so that you can choose the approach that is right for your team.
报告测试结果的方式有很多种。有一些供应商工具可以记录自动和手动结果。您的团队可能会找到一种方法来保存来自 Fit 等工具的结果,或者您可能只是选择保留一个大的可见的手动图表。
There are many ways to report test results. There are vendor tools that will record both automated and manual results. Your team may find a way to persist the results from tools such as Fit, or you may just choose to keep a big visible manual chart.
一些团队采用的方法是创建本地测试结果应用程序。例如,用Ruby on Rails 为数据库编写的一个简单的Ruby 应用程序或一个带有PHP 前端的MySQL 数据库都可以构成一个非常简单但易于使用的测试管理系统。
The approach that a few teams have taken is to create home-grown test result applications. For example, a simple Ruby application written with Ruby on Rails for the database or a MySQL database with a PHP front end can make a very simple but easy-to-use test management system.
诸如此类的工具可以非常简单,也可以包含额外的复杂性,例如对测试进行分类的功能。重要的是测试结果。如果您的自动化测试记录了它们通过或失败的结果以及错误,您就有一些历史可以帮助确定测试的脆弱性。
A tool such as this can be very simple or can include added complexity such as the capability to categorize your tests. The important thing is the test results. If your automated tests record their pass or fail result along with the error, you have some history to help determine fragility of the test.
您的团队可以配置您的自动构建过程,以通过电子邮件或团队成员可以在线查看的反馈实用程序或 Web 界面提供每个构建的测试结果。随着时间的推移,结果可以用多种格式进行总结,使进展可见。Lisa 的一个团队制作了一份每日图表,记录了编写、运行和通过的测试,并将其张贴在团队的工作区中。另一个生成了一个日历,其中包含每天通过的单元测试的数量。即使是简单的视觉效果也是有效的。
Your team can configure your automated build process to provide test results from each build, by email, or a feedback utility or web interface that team members can view online. Results over time can be summarized in a variety of formats that make progress visible. One of Lisa’s teams produced a daily graph of tests written, run, and passing that was posted in the team’s work area. Another produced a daily calendar with the number of unit tests passing every day. Even simple visual results are effective.
我们将在本章后面讨论一些您可以使用的指标。
We talk about some of the metrics you can use later in this chapter.
我们已经讨论了为什么要测试计划以及您应该考虑什么。现在我们讨论一些您可能习惯的繁重测试计划的替代方案。无论您的组织使用何种类型的测试计划,都将其变成您的。以有利于您的团队的方式使用它,并确保您满足客户的需求。与您的团队制作的任何文档一样,它应该实现一个目的。
We’ve talked about why to test plan and what you should consider. Now we talk about some of the alternatives to the heavy test plans you may be used to. Whatever type of test plan your organization uses, make it yours. Use it in a way that benefits your team, and make sure you meet your customer’s needs. As with any document your team produces, it should fulfill a purpose.
如果您的组织或客户坚持针对 SOX 合规性或其他法规需求的测试计划,请考虑一个涵盖必需品但不包含任何额外内容的轻量级测试计划。不要重复已经包含在项目计划或项目章程中的项目。示例测试计划可能看起来像图 15-6中所示的那样。
If your organization or customer insists on a test plan for SOX compliance or other regulatory needs, consider a lightweight test plan that covers the necessities but not any extras. Do not repeat items that have already been included in the Project Plan or Project Charter. A sample Test Plan might look something like the one shown in Figure 15-6.
测试计划不应涵盖所有可能发生的情况或每个故事,也无意解决可追溯性问题。它应该是一个帮助您考虑测试项目风险的工具。它不应取代与客户或团队其他成员的面对面交谈。
A test plan should not cover every eventuality or every story, and it is not meant to address traceability. It should be a tool to help you think about testing risks to your project. It should not replace face-to-face conversation with your customer or the rest of your team.
Janet 使用发布计划与测试人员和客户一起开发高级测试矩阵。测试矩阵是一种简单的方式来传达有关您要测试的功能的大局。它可以让您的团队快速了解所需的测试。
Janet uses release planning to work with the testers and customers to develop a high-level test matrix. A test matrix is a simple way to communicate the big picture concerning what functionality you want to test. It gives your team a quick overview of the testing required.
测试矩阵只是侧面的功能列表和顶部的测试条件。在考虑测试条件和功能时,请考虑整个应用程序以及新功能或更改功能的任何影响可能对应用程序的其余部分有影响。测试人员与客户坐在一起思考测试条件是很重要的。
A test matrix is just a list of functionality down the side and test conditions across the top. When thinking about test conditions and functionality, consider the whole application and any impact the new or changed functionality might have on the rest of the application. Testers sitting with customers and thinking about test conditions is what is important.
它也可以是一种跟踪覆盖率的机制,并且可以尽可能详细。团队可以使用高级测试矩阵向客户团队或管理层展示已经测试过的内容和剩余的内容。团队可以使用更详细的测试矩阵来显示测试计划和跟踪测试进度。创建矩阵后,在完成测试时填充方块变得很容易。把事情简单化。因为我们喜欢易于阅读的大图表,所以我们推荐对您的团队有意义的颜色。例如,绿色 (G) 表示测试已完成并且团队对此感到满意,而黄色 (Y) 可能表示已经完成了一些测试,但如果有时间还需要进行更多的探索性测试。红色 (R) 表示有东西坏了。白色方块表示尚未测试,
It can also be a mechanism to track coverage and can be as detailed as you like. A high-level test matrix can be used by the team to show the customer team or management what has been tested already and what is left. A more detailed test matrix can be used by the team to show what is planned for testing and track the progress of the testing. After the matrix has been created, it becomes easy to fill in the squares when testing is done. Keep it simple. Because we like big visible charts that are easy to read, we recommend colors that mean something to your team. For example, green (G) means testing is done and the team is happy with it, while yellow (Y) might mean some testing has been done but more exploratory testing is needed if there is time. Red (R) means something is broken. A white square means it hasn’t been tested yet, and a gray (not applicable) square means it doesn’t need to be tested.
让我们看一个例子。我们有一个小版本要发布,用于计算运费。在图 15-7中,不同的功能部分在一个轴上表示,而装运的属性在另一个轴上表示。单个单元格采用颜色编码,以显示哪些案例已经过测试,哪些需要更多关注。“<= 2 lbs”的所有单元格都已完成,> 4 lbs 的前三个单元格已完成但需要更多探索性测试,“Ship to Alaska”/“>4 lbs”单元格表示可能存在问题。
Let’s look at an example. We have a small release we want to put out that calculates shipping costs. In Figure 15-7, different pieces of functionality are represented on one axis, and properties of the shipment are represented on the other. Individual cells are color-coded to show which cases are tested and which need more attention. All of the cells for “<= 2 lbs” are finished, the top three cells for > 4 lbs are done but need more exploratory testing, and the “Ship to Alaska”/“>4 lbs” cell denotes a possible issue.
测试矩阵是一个非常强大的工具,如果您的团队有这些问题,可用于帮助解决可追溯性问题。考虑什么对您的团队有意义,并根据您的团队和您自己的情况进行调整。
A test matrix is a very powerful tool and can be used to help address traceability issues if your team has those problems. Think about what makes sense for your team and adapt it for your team and what makes sense to you.
珍妮特还看到了电子表格格式的使用并取得了一些成功。例如,在 WestJet,工作簿中的第一个选项卡是应用程序中存在的高级功能列表。对于每一行,团队确定该项目是否影响了该功能。如果是这样,他们给出了预期影响的评级。在确定更改的影响之后,就可以做出有关测试环境、测试数据或 UAT 的决策。
Janet has also seen a spreadsheet format used with some success. For example, at WestJet, the first tab in a workbook was a high-level list of functionality that existed in the application. For each row, the team determined if the project affected that piece of functionality. If so, they gave a rating of the expected impact. After the impact of the changes had been determined, decisions about test environments, test data, or UAT could then be made.
选项卡用于风险和假设,但可用于您的团队可能需要的任何内容。电子表格等灵活的格式意味着您可以对其进行定制以适合您。
Tabs were used for risks and assumptions but could be used for anything your team may need. A flexible format such as a spreadsheet means you can tailor it to work for you.
该信息可以多种不同方式使用。它可用于确定将探索性测试工作集中在何处,或者可能有助于创建高级测试矩阵以确保您在测试期间接触到所有区域。
This information can be used in a number of different ways. It can be used to determine where to concentrate your exploratory testing efforts, or maybe to help create a high-level test matrix to make sure you touch on all of the areas during your testing.
如果您的团队是非正式的并且发布的版本很少,那么任何类型的文档都可能太多了。有时在白板或索引卡上列出风险和假设就足够了。珍妮特使用白板来管理风险,它运作良好。如果风险真的成为一个问题,结果会被记录下来并划掉。添加新的风险和缓解策略很容易,而且整个团队都可以看到该列表。这也可以在 wiki 页面上完成。
If your team is informal and has small releases, any kind of documentation may be too much. Sometimes it’s enough to list the risks and assumptions on a whiteboard or on index cards. Janet has used a whiteboard to manage risks, and it worked quite well. If a risk actually became an issue, the result was documented and crossed off. It was easy to add new risks and mitigation strategies, and the list was visible to the whole team. This could also be done on a wiki page.
我们怎么强调您都需要了解您的团队及其需求。
We cannot stress enough that you need to know your team and its needs.
有时您可能需要向客户提供更多信息,例如测试用例列表。如果您的团队有一个工具,您可以从中提取测试用例名称列表,您可以轻松地将此列表提供给需要它的任何人。这将提供更多传统类型的详细测试计划,但只有在实际编写测试后才能使用。我们不建议在这上面花任何时间,因为我们看不到附加值,但有时可能需要此列表进行风险评估或审计。
Sometimes you may be required to present more information to your customers, such as a list of test cases. If your team has a tool from which you could extract a list of test case names, you could provide this list easily to anyone who needed it. This would present more of a traditional type detailed test plan but wouldn’t be available until after tests were actually written. We don’t recommend spending any time on this because we don’t see added value, but sometimes this list may be required for risk assessment or auditability.
如果您的团队刚刚开始敏捷开发,请确保为早期迭代准备好必要的基础设施。您可以在进行过程中改变跟踪进度的方式,而您的回顾将帮助您发现这些问题。如果您在完成为每次迭代计划的工作时遇到问题,也许您需要更多可见的图表或视觉辅助工具来帮助您衡量进度并进行迭代中期调整。您的客户是否有办法了解迭代的进展情况以及完成了哪些故事?在每次迭代之前花点时间评估您是否获得了正确的反馈以跟踪测试。
If your team is just getting started with agile development, make sure you have necessary infrastructure in place for your early iterations. You may change the way you are tracking progress as you go along, and your retrospectives will help you bring these issues to light. If you’re having problems completing the work planned for each iteration, maybe you need more visible charts or visual aids to help you gauge progress and make mid-iteration adjustments. Do your customers have some way to know how the iteration is progressing and which stories are done? Take time before the each iteration to evaluate whether you’re getting the right kind of feedback to keep track of testing.
我们所知道的高效敏捷团队都遵循这个简单的规则:“故事只有经过测试才能完成。” 这条规则可以扩展为不仅必须测试故事,还必须签入代码,它必须具有由持续构建过程运行的自动化测试,它必须被记录下来,或者任何你团队的“完成”标准是。在迭代期间的任何时候,您都需要能够快速评估每个故事还剩下多少测试工作,以及哪些故事已经“完成”。故事板或任务板是实现此目的的理想选择,尤其是当它们使用颜色编码来表示测试任务与开发和其他类型的任务时。软木板、带磁铁的钢板、海报大小的便利贴或白板都可以。给每个故事单独一行,并按优先级排序。有“待办事项”、“进行中的工作”、“验证、
The effective agile teams we know all follow this simple rule: “No story is done until it’s tested.” This rule can be expanded to say that not only must the story be tested, the code must be checked in, it must have automated tests that are run by a continual build process, it must be documented, or whatever your team’s “doneness” criteria are. At any time during an iteration, you need to be able to quickly assess how much testing work remains on each story, and which stories are “done.” Story or task boards are ideal for this purpose, especially if they use color-coding to denote test tasks vs. development and other types of tasks. Cork boards, steel sheets with magnets, poster-sized sticky notes, or whiteboards all work fine. Give each story its own row, and order them by priority. Have columns for “to do,” “work in progress,” “verify,” and “done.”
一些团队针对不同类型的任务使用不同颜色的索引卡:绿色用于测试,白色用于编码,黄色和红色用于错误。其他团队每个开发任务使用一张卡片,并添加不同颜色的贴纸以表明测试正在进行或表明有错误需要解决。使用任何方法,让您快速了解有多少故事已“完成”,所有编码、数据库、测试和其他任务都已完成。随着迭代的进行,很容易看出团队是否走上正轨,或者您是否需要提出一个故事或让程序员参与测试任务。
Some teams use different colored index cards for the different types of tasks: green for testing, white for coding, yellow and red for bugs. Other teams use one card per development task, and add different colored stickers to show that testing is in progress or show that there are bugs to resolve. Use any method that lets you see at a quick glance how many stories are “done,” with all coding, database, testing, and other tasks completed. As the iteration progresses, it’s easy to see if the team is on track, or if you need to pull a story out or have programmers pitch in on testing tasks.
分布式团队需要某种在线故事板。这可能是一个电子表格,或者像 Mingle 那样模仿物理故事板的专用软件。
Distributed teams need some kind of online story board. This might be a spreadsheet, or specialized software that mimics a physical story board as Mingle does.
早些时候,我们谈到了计划如何跟踪测试结果。现在我们想谈谈如何有效地与他们沟通。测试结果是衡量进度的最重要方法之一,查看是否正在为每个故事编写和运行新测试,以及它们是否全部通过。一些团队会发布大的可见图表,显示已编写、运行和通过的测试数量。其他人让他们的构建过程通过电子邮件将自动测试结果发送给团队成员和利益相关者。一些持续集成工具提供 GUI 工具来监控构建和构建结果。
Earlier, we talked about planning how to track test results. Now we want to talk about effectively communicating them. Test results are one of the most important ways to measure progress, see whether new tests are being written and run for each story, and whether they’re all passing. Some teams post big visible charts of the number of tests written, run, and passed. Others have their build process email automated test results to team members and stakeholders. Some continuous integration tools provide GUI tools to monitor builds and build results.
我们听说有团队将投影仪连接到机器上,在连续构建中运行 FitNesse 测试并始终显示测试结果。测试结果是团队进步的具体描述。如果数量测试不会每天或每次迭代都进行,这可能表明存在问题。要么团队没有编写测试(假设他们正在开发测试优先),要么他们没有完成很多代码。当然,他们可能会删除旧代码和随之而来的测试。重要的是要分析趋势为何走向错误的方向。下一节将提供一些关于您可能希望收集和显示的指标类型的想法。
We’ve heard of teams that have a projector hooked up to the machine that runs FitNesse tests on a continuous build and displays the test results at all times. Test results are a concrete depiction of the team’s progress. If the number of tests doesn’t go up every day or every iteration, that might indicate a problem. Either the team isn’t writing tests (assuming they’re developing test-first), or they aren’t getting much code completed. Of course, it’s possible they are ripping out old code and the tests that went with it. It’s important to analyze why trends are going the wrong way. The next section gives you some ideas about the types of metrics you may want to gather and display.
无论您的团队决定他们想要传达您的进度,请确保您事先考虑好,并且每个人都能从中获益。
However your team decides they want to communicate your progress, make sure you think about it up front and everyone gets value from it.
我们在此处包含此部分,因为了解您希望从发布一开始就收集哪些指标非常重要。这些指标应该为您提供有关开发进展情况的持续反馈,以便您可以响应意外事件并根据需要更改您的流程。请记住,您需要了解您试图用您的指标解决什么问题,以便您可以跟踪正确的问题。我们在这里讨论的指标只是您可以选择跟踪的一些示例。
We include this section here, because it is important to understand what metrics you want to gather from the very beginning of a release. These metrics should give you continual feedback about how development is proceeding, so that you can respond to unexpected events and change your process as needed. Remember, you need to understand what problem you are trying to solve with your metrics so that you can track the right ones. The metrics we talk about here are just some examples that you may choose to track.
许多敏捷团队跟踪每个级别的测试数量:单元、功能、故事测试、GUI、负载等。趋势比数字更重要。看到测试数量增加,我们会有一种温暖而模糊的感觉。不过,没有上下文的数字只是一个数字。例如,如果一个团队说它有 1000 个测试,那是什么意思?1000 次测试的覆盖率是 10% 还是 90%?删除具有测试的代码时会发生什么?
Many agile teams track the number of tests at each level: unit, functional, story tests, GUI, load, and so on. The trend is more important than the number. We get a warm fuzzy feeling seeing the number of tests go up. A number without context is just a number, though. For example, if a team says it has 1000 tests, what does that mean? Do 1000 tests give 10% or 90% coverage? What happens when code that has tests is removed?
跟踪在故事级别编写、运行和通过的测试的数量是显示故事状态的一种方式。编写的测试数量显示了驱动开发的测试进度。了解有多少测试未通过,可以让您了解还需要编写多少代码。
Tracking the number of tests written, running, and passing at a story level is one way to show a story’s status. The number of tests written shows progress of tests to drive development. Knowing how many tests aren’t passing yet gives you an idea of how much code still needs to be written.
测试通过后,只要代码中存在功能,它就需要保持“绿色”。通过和失败的测试数量随时间变化的图表显示了回归失败是否存在问题,还显示了代码库的增长。同样,重要的是趋势。注意异常。
After a test passes, it needs to stay “green” as long as the functionality is present in the code. Graphs of the number of tests passing and failing over time show whether there’s a problem with regression failures and also show the growth of the code base. Again, it’s the trend that’s important. Watch for anomalies.
这些类型的测量可以简单地报告并且仍然有效。
These types of measurements can be reported simply and still be effective.
有不同的方法来衡量测试的数量。选择一个并尝试在所有类型的测试中保持全面一致,否则您的指标可能会变得混乱。衡量测试脚本或类的数量是一种方法,但每个都可能包含多个单独的测试用例或“断言”,因此计算这些可能更准确。
There are different ways to measure the number of tests. Choose one and try to stay consistent across the board with all types of tests, otherwise your metrics may get confusing. Measuring the number of test scripts or classes is one way, but each one may contain multiple individual test cases or “asserts,” so it may be more accurate to count those.
如果您要对测试进行计数,请务必报告该信息以便可以使用。构建电子邮件或构建状态 UI 可以传达不同级别的测试运行、通过和失败的数量。客户团队可能会满意仅在每个冲刺结束时、在冲刺回顾中或电子邮件中查看此信息。
If you’re going to count tests, be sure to report the information so that it can be used. Build emails or build status UIs can communicate the number of tests run, passed, and failed at various levels. The customer team may be content to see this information only at the end of each sprint, in the sprint review, or an email.
无论您选择收集什么指标,请确保团队接受它们。
Whatever metrics you choose to gather, be sure the team buys into them.
不要过于专注于实际测量结果,以至于您没有意识到趋势的其他副作用。如果需要,请对调整您正在测量的内容持开放态度。
Don’t get so caught up in the actual measurements that you don’t recognize other side effects of the trending. Be open to adjusting what you are measuring if the need is there.
代码覆盖率是另一个传统指标。我们的测试执行了多少代码?有优秀的商业和开源代码覆盖率工具可用,这些工具可以集成到您的构建过程中,以便您立即知道覆盖率是上升还是下降。与大多数指标一样,趋势是值得关注的。图 15-11显示了示例代码覆盖率报告。
Code coverage is another traditional metric. How much of our code is exercised by our tests? There are excellent commercial and open source code coverage tools available, and these can be integrated into your build process so that you know right away if coverage has gone up or down. As with most metrics, the trend is the thing to watch. Figure 15-11 shows a sample code coverage report.
图 15-12和15-13是共同作用的两个趋势示例。图 15-12显示了每次迭代的方法总数的趋势。图 15-12是匹配的代码覆盖率。这些示例说明了为什么需要在上下文中查看图表。如果您只看第一张显示方法数量的图表,您只会了解一半的情况。方法的数量在增加,看起来不错,但实际上覆盖率在下降。我们不知道覆盖率下降的原因,但这应该是问团队“为什么?”的触发因素。
Figures 15-12 and 15-13 are two examples of trends that work together. Figure 15-12 shows a trend of the total number of methods each iteration. Figure 15-12 is the matching code coverage. These examples show why graphs need to be looked at in context. If you only look at the first graph showing the number of methods, you’ll only get half the story. The number of methods is increasing, which looks good, but the coverage is actually decreasing. We do not know the reason for the decreased coverage, but it should be a trigger to ask the team, “Why?”
请记住,这些工具只能衡量您编写的代码的覆盖率。如果遗漏了某些功能,您的代码覆盖率报告将不会把它曝光。您的测试可能有 80% 的代码覆盖率,但您遗漏了本应拥有的 10% 的代码。用测试驱动开发有助于避免这个问题,但不要过度重视代码覆盖率统计数据。
Remember that these tools can only measure coverage of the code you’ve written. If some functionality was missed, your code coverage report will not bring that to light. You might have 80% code coverage with your tests, but you’re missing 10% of the code you should have. Driving development with tests helps avoid this problem, but don’t value code coverage statistics more than they deserve.
好的指标需要一些好的计划。额外的努力可以为您提供更有意义的数据。Pierre Veragen 的团队成员使用中断测试基线技术来了解他们的代码覆盖率指标是否有意义。他们手动在每种方法中引入缺陷,然后运行测试以确保测试能够捕获问题。一些测试只是确保代码返回一些值,任何值。Pierre 的团队确保测试返回正确的值。通过这种方式,他们可以确定他们的测试覆盖率是否足够好。
Good metrics require some good planning. Extra effort can give you more meaningful data. Pierre Veragen’s team members use a break-test baseline technique to learn if their code coverage metric is meaningful. They manually introduce a flaw into each method and then run their tests to make sure the tests catch the problem. Some tests just make sure the code returns some value, any value. Pierre’s team makes sure the tests return the correct value. In this way, they can determine whether their test coverage is good enough.
代码覆盖率只是难题的一小部分。就这样使用它。它不会告诉你你的测试有多好,但只有在测试期间运行了一段特定的代码。它也不会告诉您是否运行了应用程序的不同路径。了解您的应用程序并尝试确定您的最高风险区域,并为这些区域设定一个比低风险区域更高的覆盖目标。不要忘记将您的功能测试也包含在覆盖率报告中。
Code coverage is just one small part of the puzzle. Use it as such. It doesn’t tell you how good your tests are but only if a certain chunk of code was run during the test. It does not tell you if different paths through the application were run, either. Understand your application and try identifying your highest risk areas, and set a coverage goal that is higher for those areas than for low-risk areas. Don’t forget to include your functional tests in the coverage report as well.
当您的团队设定与缺陷相关的目标时,使用适当的指标来衡量实现这些目标的进度。有些趋势是您希望针对整个版本监控的,有些是特定于迭代的。例如,如果您试图实现零缺陷,您可能希望在每次迭代结束时跟踪未解决的错误,或者在开发之后但在发布之前发现了多少错误。我们大多数人都想知道在代码投入生产后报告了多少缺陷,这是完全不同的事情。这些问题会在事后告诉您您的团队在上一个版本上的表现如何,但不会告诉您您在当前版本上的表现如何。他们可能会告诉您需要更改哪些流程以减少缺陷数量。Lisa 的团队更关心在新架构中重写的“新”代码中发现的生产缺陷。他们正在努力生产零缺陷的新代码,因此他们需要知道自己的表现如何。他们预计在遗留系统中会经常发现错误,其中只有最关键的功能被自动化的 GUI 冒烟测试覆盖,并且几乎没有自动化的单元测试和 GUI 背后的测试。
As your team sets goals related to defects, use appropriate metrics to measure progress toward those goals. There are trends that you will want to monitor for the whole release, and there are ones that are iteration-specific. For example, if you’re trying to achieve zero defects, you may want to track the open bugs at the end of each iteration, or how many bugs were found after development but before release. Most of us are interested in knowing how many defects have been reported after the code is in production, which is something completely different altogether. These issues will tell you after the fact how well your team did on the last release, but not how well you are doing on the current release. They may give you some indication of what processes you need to change to reduce the number of defects. Lisa’s team is more concerned with production defects found in the “new” code that was rewritten in the new architecture. They’re working hard to produce this new code with zero defects, so they need to know how well they’re doing. They expect that bugs will be found fairly often in the legacy system, where only the most critical functionality is covered by automated GUI smoke tests, and there are few automated unit and behind-the-GUI tests.
了解遗留代码的缺陷率可能是重构或重写它的充分理由,但团队的首要任务是做好新代码,因此他们按“新”和“旧”代码对错误进行分组,并专注于“新”错误。
Knowing the defect rate of legacy code might be good justification for refactoring or rewriting it, but the team’s top priority is doing a good job with the new code, so they group bugs by “new” and “old” code, and focus on the “new” bugs.
确保您的错误数据库可以跟踪您想要衡量的内容。您可能必须对数据库和流程进行一些更改才能获得所需的数据。例如,如果你想测量发布后在生产中发现了多少缺陷,你必须确保你有环境和版本作为必填字段,或者确保输入错误的人总是填写它们。
Make sure your bug database can track what you want to measure. You may have to make some changes in both the database and your process to get the data you need. For example, if you want to measure how many defects were found in production after a release, you have to make sure you have environment and version as mandatory fields, or make sure that people who enter bugs always fill them in.
有关缺陷跟踪系统的更多信息,请参阅第 5 章“转换传统流程”。
More on defect tracking systems can be found in Chapter 5, “Transitioning Traditional Processes.”
因为缺陷跟踪系统通常用于跟踪错误以外的目的,所以一定不要混淆数字。手动更新数据库的请求不一定反映现有代码的问题。正确使用您的缺陷跟踪工具以确保您的指标有意义。
Because defect tracking systems are often used for purposes besides tracking bugs, be sure not to muddle the numbers. A request for a manual update to the database doesn’t necessarily reflect an issue with the existing code. Use your defect tracking tool properly to ensure that your metrics are meaningful.
发布计划是评估您一直跟踪的指标的投资回报率的好时机。您花费了多少精力来收集和报告指标?他们会告诉您您需要知道的信息吗?您发布的代码是否符合您团队的内部质量标准?代码覆盖率在上升吗?团队是否实现了减少投入生产的缺陷数量的目标?如果没有,是否有充分的理由?
Release planning is a good time to evaluate the ROI of the metrics you’ve been tracking. How much effort are you spending to gather and report the metrics? Do they tell you what you need to know? Does the code you release meet your team’s standards for internal quality? Is the code coverage percentage going up? Is the team meeting its goals for reducing the number of defects that get out to production? If not, was there a good reason?
指标只是其中的一块拼图。使用您的发布、主题或项目规划会议,在业务需要时重新专注于交付业务价值。花一些时间了解您将要开发的功能。不要陷入对你的计划的承诺——情况一定会改变。相反,准备进行正确的活动并及时获得正确的资源以满足客户的优先事项。
Metrics are just one piece of the puzzle. Use your release, theme, or project planning meetings to refocus on delivering business value when the business needs it. Take some time to learn about the features you’re about to develop. Don’t get caught up with committing to your plans—the situation is bound to change. Instead, prepare for doing the right activities and getting the right resources in time to meet the customers’ priorities.
当您的团队制定新主题或新版本的计划时,请牢记本章的要点。
As your team puts together its plan for a new theme or release, keep the main points of this chapter in mind.
在评估故事大小时,请考虑不同的观点,包括业务价值、风险、技术实施以及功能的使用方式。提出澄清问题,但不要陷入细节。
When sizing a story, consider different viewpoints, including business value, risk, technical implementation, and how the feature will be used. Ask clarifying questions, but don’t get bogged down in details.
测试人员可以通过功能集帮助确定“薄片”或“关键路径”,以帮助确定故事的优先级。如果高风险故事可能需要及早进行额外测试,请尽早安排。
Testers can help identify the “thin slice” or “critical path” through a feature set to help prioritize stories. Schedule high-risk stories early if they might require extra testing early.
故事测试工作的规模有助于确定该故事是否在发布范围内。
The size of testing effort for a story helps determine whether that story is in scope for the release.
测试人员可以帮助团队思考新故事将如何影响更大的系统。
Testers can help the team think about how new stories will impact the larger system.
当功能可能影响外部团队开发的系统或子系统时,计划额外的测试时间和资源。
Plan for extra testing time and resources when features may affect systems or subsystems developed by outside teams.
当团队确定发布范围时,评估测试范围并为其预算足够的时间和资源。
As the team identifies the scope of the release, evaluate the scope of testing and budget enough time and resources for it.
在发布计划期间花一些时间来解决基础设施、测试环境和测试数据问题。
Spend some time during release planning to address infrastructure, test environment, and test data concerns.
轻量级、敏捷的测试计划可以帮助确保在发布或项目的生命周期中解决所有测试注意事项。
A lightweight, agile test plan can help make sure all of the testing considerations are addressed during the life of the release or project.
考虑可能更适合您的团队的测试计划的替代方案;测试矩阵、电子表格甚至白板就足够了。
Consider alternatives to test plans that might be more appropriate for your team; test matrices, spreadsheets, or even a whiteboard may be sufficient.
正式的发布计划可能不适合您的情况。在没有发布计划的情况下,考虑至少确定和讨论应该首先完成的前几个故事。
Formal release planning may not be appropriate for your situation. In the absence of release planning, consider identifying and discussing at least the first few stories that should be done first.
计划您希望在发布生命周期内捕获哪些指标;考虑您要解决的问题并仅捕获对您的团队有意义的指标。
Plan for what metrics you want to capture for the life of the release; think about what problem you are trying to solve and capture only those metrics that are meaningful for your team.
在敏捷开发中,我们通常喜欢“及时”地完成任务。我们看不到前方道路的弯道,所以我们专注于手头的活动。再一次,我们希望在开始每次新迭代时立即开始运行。这可能需要一些准备。烘焙在这里是一个很好的类比。你决定要烤饼干,因为有人过来了。在你开始之前,你要确保你有正确的成分。如果你不这样做,你要么去买你需要的东西,要么你选择另一种来制作。
In agile development, we generally like to do tasks “just in time.” We can’t see around the curves in the road ahead, so we focus on the activities at hand. Then again, we want to hit the ground running when we start each new iteration. That may require a little preparation. Baking is a good analogy here. You decide you want to bake cookies because someone is coming over. Before you start, you make sure you have the right ingredients. If you don’t, you either go buy what you need, or you choose a different kind to make.
不要做得太过分——如果迭代前的活动在迭代期间没有节省时间,或者帮助你做得更好,那么不要在迭代前做。做适合您团队的事情,并不断尝试。也许您会在迭代开始后进行其中的一些活动。这里有一些想法可以帮助您将质量“融入”到您的产品中。
Don’t go overboard—if a pre-iteration activity doesn’t save time during the iteration, or help you do a better job, don’t do it before the iteration. Do what is appropriate for your team, and keep experimenting. Maybe you’ll do some of these activities after the iteration starts instead. Here are some ideas to think about that might help you “bake quality in” to your product.
在第 2 章“敏捷测试人员的十项原则”中,我们解释了敏捷测试人员必须如何转变他们的思维方式。我们没有等待工作来找我们,而是培养了一种积极主动的态度,我们起身去寻找方法贡献。随着敏捷开发快速而持续的步伐,很容易沉浸在当前迭代的故事中。我们忙于确保我们已经通过前期测试涵盖了这些功能,执行探索性测试以确保我们了解业务需求,并自动执行足够的回归测试,很难想到其他任何事情。然而,有时花点时间帮助我们的客户和我们的团队为下一次迭代做准备是合适的。当我们的团队即将开辟新天地,或处理复杂且有风险的故事时,在迭代之前进行一些工作可以帮助最大限度地提高团队的速度并最大限度地减少挫败感。
In Chapter 2, “Ten Principles of an Agile Tester,” we explained how agile testers have to shift their mind-set. Instead of waiting for work to come to us, we develop a proactive attitude where we get up and go look for ways to contribute. With the fast and constant pace of agile development, it’s easy to get immersed in the current iteration’s stories. We are so busy making sure we’ve covered the features with up-front tests, performing exploratory testing to be sure we’ve understood the business requirements, and automating adequate regression tests, it’s hard to think of anything else. However, it’s sometimes appropriate to take a bit of time to help our customers and our team prepare for the next iteration. When our team is about to break new ground, or work on complex and risky stories, some work before the iteration can help maximize our team’s velocity and minimize frustration.
我们肯定不想把所有时间都花在开会上,或计划可能重新确定优先级的故事。然而,如果我们能让我们的迭代计划进行得更快,并降低我们将要承担的故事的风险,那么在我们开始迭代之前进行一些研究和集思广益是值得的。
We sure don’t want to spend all our time in meetings, or planning for stories that might be re-prioritized. However, if we can make our iteration planning go faster, and reduce the risk of the stories we’re about to undertake, it’s worth doing some research and brainstorming before we start the iteration.
在迭代之前处理故事可能对分散在不同地理位置的团队特别有用。通过提前工作,有时间向每个人提供信息并让他们有机会提供意见。
Working on stories in advance of the iteration may be especially useful for teams that are split across different geographic locations. By working ahead, there’s time to get information to everyone and give them a chance to give their input.
问题是我们在每次短暂的敏捷迭代中都非常忙碌,很难抽出时间来讨论下一次迭代的故事,更不用说开始编写测试用例了。如果您的迭代总是顺利进行,故事递增地交付并且有足够的时间进行测试,并且交付的软件符合客户的期望,那么您可能不需要花时间提前准备。如果您的团队在完成故事时遇到困难,或者最终发现功能的实际行为与期望行为之间存在很大的不匹配,那么提前一点计划可能会在迭代期间为您节省时间。
The problem is that we’re so busy during each short agile iteration that it’s hard to find time to meet about the next iteration’s stories, much less start writing test cases. If your iterations always go smoothly, with stories delivered incrementally and plenty of time to test, and the delivered software matches customer expectations, you may not need to take time to prepare in advance. If your team has trouble finishing stories, or ends up with big mismatches between actual and desired behavior of features, a little advance planning may save you time during the iteration.
您的团队可能不需要太多或任何提前准备。Pierre Veragen 和 Erika Boyer 向 Lisa 描述了他们在 iLevel by Weyerhaeuser 的团队如何在他们的迭代启动会议上一起编写用户验收测试。
Your team may not need much or any advance preparation. Pierre Veragen and Erika Boyer described to Lisa how their teams at iLevel by Weyerhaeuser write user acceptance tests together at their iteration kickoff meeting.
这些测试与故事叙述一起写在维基页面或类似工具上,稍后在团队成员为每个故事编写任务卡并开始编写更多测试和代码时使用。例子变成了可执行的测试。因为随着团队对故事的了解越来越多,测试也会发生变化,所以团队可能会选择不维护开始时编写的原始测试。开始时保持简单,稍后再深入研究细节。
These tests, which were written on a wiki page or some similar tool along with the story narrative, are used later when team members write task cards for each story and start writing more tests and code. Examples are turned into executable tests. Because the tests change as the team learns more about the story, the team may opt not to maintain the original ones that were written at the start. Keep it simple to start with, and dig into details later.
丽莎随后观察了他们的一次计划会议,并亲眼目睹了这项技术的有效性。即使产品经理提供了具体的示例,将它们变成测试也可能会消除遗漏的需求。他们的团队在迭代计划会议之前不需要这样做,但并非所有团队都是这样。
Lisa subsequently observed one of their planning sessions and saw first-hand how effective this technique was. Even when the product manager provides concrete examples, turning them into tests may flush out missing requirements. Their team did not need to do this before the iteration planning session, but it is not the case with all teams.
尝试进行简短的预迭代讨论和测试写作会议。您需要多次迭代才能找到团队的节奏,并了解提前讨论故事是否能让您在迭代期间更有效率。
Experiment with short pre-iteration discussions and test-writing sessions. It’ll take you several iterations to find your team’s rhythm, and find out if advance story discussions make you more productive during the iteration.
“提前工作”存在风险。您可以花时间了解有关某个功能的更多详细信息,结果业务人员在最后一刻重新确定优先级并无限期地推迟该功能。在适当的时候投入准备时间。当你知道你有一个复杂的主题或故事即将到来,并且它有一个严格的截止日期,就像 Lisa 的团队对声明故事所做的那样,考虑花一些时间预先检查不同的观点。提前讨论故事的唯一原因是为了在迭代计划和开发过程中节省时间。更深入地了解功能行为可以加快测试和编码速度,并有助于确保您提供正确的功能。
There’s a risk to “working ahead.” You could spend time learning more details about a feature only to have the business people re-prioritize at the last minute and put that feature off indefinitely. Invest preparation time when it’s appropriate. When you know you have a complex theme or story coming up, and it has a hard deadline such as Lisa’s team had with the statement story, consider spending some time up front checking out different viewpoints. The only reason to discuss stories in advance is to save time during iteration planning and during development. A deeper understanding of the feature behavior can speed up testing and coding, and can help make sure you deliver the right functionality.
如果您的情况如此多变以至于故事可能会在迭代开始的那一天重新确定优先级,那么不值得尝试进行此计划。相反,请确保在计划会议期间为这些讨论预留时间。
If your situation is so dynamic that stories might be re-prioritized the day that the iteration starts, it isn’t worth trying to do this planning. Instead, make sure you budget time for these discussions during your planning meeting.
Lisa 的产品负责人 Steve Perkins 提出了“预先清晰度”这个词。每个组织的不同部分有不同的优先事项和议程。例如,业务开发部正在寻找新功能来吸引新业务,而运营部则优先考虑可以减少用户来电次数的功能。开发团队试图了解业务需求的范围并了解每个人的工作。
Lisa’s product owner, Steve Perkins, came up with the term “advance clarity.” Different parts of each organization have different priorities and agendas. For example, Business Development is looking for new features to attract new business, while Operations is prioritizing features that would reduce the number of phone calls from users. The development team tries to understand the range of business needs and get a feel for each individual’s job.
对于许多不同的议程,需要有人来决定在下一次迭代中应该实施哪些故事。因为有很多方法可以实现任何给定的故事,所以必须有人决定具体的需求,并以示例、满足条件和测试用例的形式捕获它们。史蒂夫让每个人都聚在一起,就他们希望从每个故事中获得的价值达成一致,并提供“预先的清晰度”。
With many different agendas, someone needs to decide what stories should be implemented in the next iteration. Because there are many ways to implement any given story, someone has to decide the specific requirements and capture them in the form of examples, conditions of satisfaction, and test cases. Steve gets everyone together to agree on the value they want from each story, and to provide “advance clarity.”
Scrum 提供了产品负责人的有用角色,以帮助所有客户“用同一个声音说话”。无论你是否在 Scrum 团队中,找到一些方法来帮助你的客户就故事的优先级以及如何每个故事的组成部分都应该被实现。管理层的支持是至关重要的,因为担任此角色的任何人都需要时间和权力来让所有人达成共识。
Scrum provides the helpful role of the product owner to help all the customers “Speak with One Voice.” Whether or not you’re on a Scrum team, find some way to help your customers agree on the priority of the stories and how the components of each story ought to be implemented. Management support is crucial, because any person in this role needs time and authority to get everyone on the same page.
其他团队使用业务分析师在下一次迭代之前帮助充实故事。在 Janet 工作的一个组织中,客户无法全职回答问题,但每个团队都有一名业务分析师与客户合作,在迭代计划会议之前充实需求。如果有任何她在会议上无法回答的问题,团队要么直接给客户打电话,要么分析师会后立即跟进。
Other teams use business analysts to help flesh out the stories before the next iteration. In one organization Janet worked with, the customers were not available full-time to answer questions, but each team had a business analyst that worked with the customers to flesh out the requirements before the iteration planning meeting. If there were any questions that she could not answer at the meeting, the team either called the customer directly or the analyst followed up immediately after the meeting.
作为一名测试人员,您想参加故事编写和优先级确定会议。提出有助于客户专注于核心功能的问题,即他们需要的关键业务价值。帮助参与者将注意力集中在具体的例子上,这些例子使故事的含义具体化。在涉及多个客户的会议中,拥有强大的协调人和确定共识的方法至关重要。
As a tester, you want to sit in on story writing and prioritization meetings. Ask questions that help the customers focus on the core functionality, the critical business value they need. Help participants stay focused on concrete examples that crystallize the meaning of the stories. In meetings that involve multiple customers, it is critical to have a strong facilitator and a method for determining consensus.
与代码一样,故事只有最少的内容才是最好的。例如,互联网购物车需要一些方法来删除不需要的物品,但是将物品从购物车移动到“稍后保存”列表的能力可能需要等待。在迭代之前讨论这个可能会有所帮助,这样团队就清楚需要计划哪些任务。首先关注最简单的事情,并用一个例子来说明。
As with code, stories are best if they have the bare minimum. For example, an Internet shopping cart needs some way to delete unwanted items, but the ability to move items from the cart to a “save for later” list can probably wait. It may be helpful to talk about this before the iteration, so that the team is clear on what tasks need to be planned. Focus on the simplest thing first and use an example to make it clear.
从不同的客户那里获取对一个故事的需求,每个客户都有不同的议程,可能会造成混乱。这就是为什么客户团队中的某个人必须达成共识并协调所有观点。这并不意味着我们不应该从不同的客户那里获得意见。作为测试人员,您要从多个角度考虑每个故事。它有助于了解故事对不同角色的人意味着什么。
Getting requirements from different customers for a story, each of whom has a different agenda, might create chaos. That’s why it’s essential for someone on the customer team to get consensus and coordinate all points of view. This doesn’t mean we shouldn’t get input from different customers. As a tester, you’re considering each story from multiple points of view. It helps to know what the story means to people in different roles.
确保您尽可能高效地收集这些数据。有时整个团队了解需求很重要,有时一两个团队成员进行研究就足够了。
Make sure you are as efficient as possible in collecting this data. Sometimes it is important for the whole team to understand the need, and sometimes it is sufficient for one or two of the team members to do the research.
当您与客户团队成员讨论下一次迭代的故事时,提出问题以帮助他们确保每个故事都能提供所需的价值。这是确定他们可能需要写的新故事的好时机。即使团队之前确定了故事的大小,您也可能会发现故事比之前想象的要大。您甚至可能会发现某个功能的实现可能比计划的更简单,故事也可能更小。
As you discuss stories for the next iteration with the customer team members, ask questions to help them make sure each story delivers the value needed. This is a good time to identify new stories they might need to write. Even though the team sized the stories previously, you might find a story is bigger than previously thought. You might even discover that a feature can be implemented more simply than planned, and the story can be smaller.
有时,当故事规模大时,会做出假设,但经过进一步调查,结果证明是错误的。即使是简单的故事也值得仔细研究。任何人都很难记住一份申请的所有细节。
Sometimes assumptions are made when the story is sized and on further investigation turn out to be false. Even simple stories deserve a closer look. It’s hard for any one person to remember all the details of an application.
这些故事表明,预先提出几个问题可能会在迭代期间节省时间,而这些时间可以用来弄清楚如何处理新发现。然而,我们认识到并非所有发现都能及早发现。例如,在第一个故事中,一个关于语句大小的简单问题可能避免了最后一刻关于如何处理四页语句的混淆,但在编码或测试之前可能没有考虑不准确的地址问题。
These stories show that a few questions up front might save time during the iteration that could be spent figuring out what to do with new discoveries. However, we recognize that not all discoveries can be found early. For example, on the first story, a simple question about statement size may have prevented last-minute confusion about how to handle four-page statements, but the inaccurate address issue may not have been considered until it was being coded or tested.
我们知道一路上总会有发现,但如果我们能首先抓住大的“陷阱”,那将有助于团队尽可能有效地工作。
We know there will always be discoveries along the way, but if we can catch the big “gotchas” first, that will help the team work as effectively as possible.
为下一次迭代做一些准备可能对分散在不同地点的团队有用。分布在多个地点的团队可以通过电话会议、在线会议或电话会议来进行迭代计划。Lisa 的一个团队使用的一种做法是给每个团队分配一个即将发生的故事的子集,并让他们提前写任务卡。在计划会议期间,每个人都可以查看所有任务卡并根据需要进行更改。前期工作加强了沟通,使故事和任务对每个人都可见,并加快了规划过程。
Some preparation for the next iteration may be useful for teams that are split across different locations. Teams that are distributed in multiple locations may do their iteration planning by conference call, online meeting, or teleconference. One practice, which a team of Lisa’s used, is to assign each team a subset of the upcoming stories and have them write task cards in advance. During the planning meeting, everyone can review all the task cards and make changes as needed. The up-front work enhances communication, makes the stories and tasks visible to everyone, and speeds up the planning process.
当然,这是假设团队正在使用电子故事板或任务板。Lisa 的团队使用 Thoughtwork 的 Mingle,但还有许多其他产品可用于此目的。
Of course, this assumes that the team is using an electronic story or task board. Lisa’s team uses Thoughtwork’s Mingle, but there are many other products out there that serve this purpose.
如果客户不能随时回答问题和做出决定,则应授权随时可以访问的其他领域专家通过确定优先级和表达所需系统来指导团队行为与例子。测试人员和业务分析师经常被要求进行这些活动。
If customers aren’t readily available to answer questions and make decisions, other domain experts who are accessible at all times should be empowered to guide the team by determining priorities and expressing desired system behavior with examples. Testers and business analysts are often called upon to do these activities.
您可能会注意到,我们几乎在本书的每一章中都讨论了示例。示例是了解和说明所需(和不需要)功能的有效方式;在整个开发周期中使用它们是值得的。我们的座右铭是布赖恩·马里克 (Brian Marick) 创造的:“一个例子现在就派上用场了。” (参见图 16-2。)用一个真实的例子开始你对特征和故事的讨论。这个想法已经流行起来,因此在最近的功能测试研讨会上,我们讨论了将其称为“示例驱动开发”的想法。
You may notice that we talk about examples in just about every chapter of this book. Examples are an effective way to learn about and illustrate desired (and undesired) functionality; it’s worth using them throughout your development cycle. Our motto was coined by Brian Marick: “An example would be handy right about now.” (See Figure 16-2.) Start your discussions about features and stories with a realistic example. The idea has taken off, so that at a recent workshop for functional testing we were discussing ideas around calling it “Example-Driven Development.”
当 Lisa 的团队成员与他们的产品负责人会面讨论下一个迭代时,他们要求他提供每个故事所需行为的示例。这使讨论保持在一个具体的层面上,并且是了解新功能应该如何工作的快速方法。执行此操作时手边有一块白板,然后开始绘图。如果一些团队成员在很远的地方,可以考虑使用允许每个人都看到白板图并参与讨论的工具。与您的客户或他们的代理人一起研究真实的例子。在发布计划期间,考虑不同的观点:业务、最终用户、开发人员和业务合作伙伴。与发布计划不同,您正在查看更多细节,因为这些是您为下一次迭代计划的故事。
When Lisa’s team members meet with their product owner to talk about the next iteration, they ask him for examples of desired behavior for each story. This keeps the discussion at a concrete level and is a fast way to learn how the new features should work. Have a whiteboard handy while you do this, and start drawing. If some team members are in a distant location, consider using tools that allow everyone to see whiteboard diagrams and participate in the discussion. Go through real examples with your customers or their proxies. As during release planning, consider different points of view: the business, end users, developers, and business partners. Unlike release planning, you are looking at far more detail because these are the stories you are planning for the next iteration.
使用示例,您可以编写高级测试来充实每个故事。您可能不需要在迭代开始之前执行此操作,但对于复杂的故事,提前编写至少一个快乐路径和一个消极路径测试用例可能是个好主意。让我们考虑一下图 16-3中的故事。
Using examples, you can write high-level tests to flesh out each story a bit more. You may not need to do this before the iteration starts, but for complex stories, it can be a good idea to write at least one happy path and one negative path test case in advance. Let’s consider the story in Figure 16-3.
产品负责人在白板上勾勒出所需的 UI。每个项目旁边都有一个“删除”复选框和一个“更新购物车”按钮。用户可以选择一个或多个项目并单击按钮删除项目。高级测试可能是:
The product owner sketches out the desired UI on the whiteboard. There’s a “delete” checkbox next to each item and an “update cart” button. The user can select one or more items and click the button to remove the items. The high-level tests might be:
当用户点击项目旁边的删除复选框并点击“更新购物车”按钮时,页面刷新显示该项目不再在购物车中。
When the user clicks the delete checkbox next to the item and clicks the “update cart” button, the page refreshes showing the item is no longer in the cart.
当用户单击购物车中每个项目旁边的删除复选框并单击“更新购物车”按钮时,页面刷新显示一个空购物车。(这会产生问题——用户应该被引导到另一个页面吗?应该显示“继续购物”按钮吗?)
When the user clicks the delete checkboxes next to every item in the cart and clicks the “update cart” button, the page refreshes showing an empty cart. (This will generate questions—should the user be directed to another page? Should a “keep shopping” button display?)
当用户单击“更新购物车”按钮而没有选中要删除的项目时,页面会刷新并且不会从购物车中删除任何内容。
When the user clicks the “update cart” button without checking an item for delete, the page is refreshed and nothing is removed from the cart.
要求您的客户在迭代之前写下示例和高级测试用例。这可以帮助他们更多地思考故事并帮助定义他们的满意度条件。它还可以帮助他们确定哪些功能是关键的,哪些可以等待。它还有助于定义故事何时完成并管理团队之间的期望。
Ask your customers to write down examples and high-level test cases before the iteration. This can help them think through the stories more and help define their conditions of satisfaction. It also helps them identify which features are critical, and which might be able to wait. It also helps to define when the story is done and manage expectations among the team.
图 16-4显示了一个示例模型,其中产品所有者在现有页面上标记了更改。使用旧系统的现有屏幕截图时要小心,因为您将冒着让新系统看起来与旧系统完全一样的风险,即使这不是您想要的。
Figure 16-4 shows a sample mock-up, where the product owner marked changes on the existing page. Be careful about using an existing screenshot from an old system, because you will run the risk of having a new system look exactly like the old one even if that is not what you wanted.
模型对于涉及 UI 或报告的故事至关重要。请您的客户就页面的外观提出他们的想法。与团队分享这些想法。一种想法是扫描它们并将它们上传到 wiki,这样每个人都可以访问。使用这些作为起点并制作更多纸质原型,或将它们画在白板上。这些可以拍照并上传以供远程团队成员查看。
Mock-ups are essential for stories involving the UI or a report. Ask your customers to draw up their ideas about how the page should look. Share these ideas with the team. One idea is to scan them in and upload them on the wiki so everyone has access. Use those as a starting point and do more paper prototypes, or draw them on the whiteboard. These can be photographed and uploaded for remote team members to see.
当您了解下一次迭代的故事时,请考虑如何接近测试它们。他们是否提出了任何特殊的自动化挑战?是否需要任何新工具?
As you learn about the stories for the next iteration, think about how to approach testing them. Do they present any special automation challenge? Are any new tools needed?
当您的团队着手开发一种新型软件时,您可能会决定进行一次开发高峰,看看您可以从中学到什么来开发它。同时,尝试测试尖峰以帮助确保您知道如何通过测试推动开发以及如何测试生成的软件。如果即将出现一个重要的新史诗或功能,写一些卡片来研究它并提前一两次举行头脑风暴会议。这有助于您在实际开始编码时了解要计划哪些故事和任务。一个想法是拥有一个“侦察”团队,看看哪些技术解决方案可能适用于即将到来的故事或主题。
When your team embarks on a new type of software, you may decide to do a development spike to see what you can learn about how to develop it. At the same time, try a test spike to help make sure you’ll know how to drive the development with tests and how to test the resulting software. If a major new epic or feature is coming up, write some cards to research it and hold brainstorming meetings an iteration or two in advance. That helps you know what stories and tasks to plan when you actually start coding. One idea is to have a “scout” team that looks at what technical solutions might work for upcoming stories or themes.
第 9 章“支持团队的面向业务的测试工具包”、第 10 章“评价产品的面向业务的测试”和第 11 章“使用面向技术的测试来评价产品”,提供了工具示例不同类型的测试。
Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” Chapter 10, “Business-Facing Tests that Critique the Product,” and Chapter 11, “Critiquing the Product Using Technology-Facing Tests,” provide examples of tools for different types of testing.
在我们的理想世界中,我们希望在每次迭代结束时零缺陷,并且绝对是在发布结束时。然而,我们认识到我们并不生活在一个理想的世界中。有时我们需要担心遗留系统缺陷,有时修复缺陷的价值不足以让企业修复。这些缺陷会发生什么?我们将在第 18 章“编码和测试”中讨论策略,但现在,我们只考虑我们有缺陷要处理。
In our ideal world, we want zero defects at the end of each iteration and definitely at the end of the release. However, we recognize that we don’t live in an ideal world. Sometimes we have legacy system defects to worry about, and sometimes fixing a defect is just not high enough value for the business to fix. What happens to these defects? We’ll talk about strategies in Chapter 18, “Coding and Testing,” but for now, let’s just consider that we have defects to deal with.
在下一次迭代之前,最好是与客户一起审查未解决的问题,并对修复与将它们留在系统中的价值进行分类。那些被认为有必要修复的应该安排到下一个迭代中。
Before the next iteration is an ideal time to review outstanding issues with the customer and triage the value of fixing versus leaving them in the system. Those that are deemed necessary to be fixed should be scheduled into the next iteration.
在迭代之前要仔细检查的另一件事是您的团队是否拥有完成任何高风险故事所需的所有资源。您需要与其他项目共享的专家吗?例如,如果其中一个故事存在安全风险或涉及安全功能,您可能需要一名安全专家。如果要进行负载测试,您可能需要有一个特殊的工具,或者需要来自另一个团队的负载测试专家,甚至是提供负载测试服务的供应商的帮助。这是您提前计划的最后机会。
Another thing to double-check before the iteration is whether your team has all the resources you need to complete any high-risk stories. Do you need any experts who are shared with other projects? For example, you may need a security expert if one of the stories poses a security risk or is for a security feature. If load testing will be done, you may need to have a special tool, or have help from a load testing specialist from another team, or even a vendor who provides load testing services. This is your last chance to plan ahead.
您的团队可能需要也可能不需要在迭代之前做任何准备。因为优先级在敏捷开发中变化很快,所以您不想浪费时间计划可能被推迟的故事。但是,如果您即将实施一些新技术,开始一个复杂的新主题,希望在迭代计划中节省时间,或者您的团队被分配到不同的地点,您可能会发现一些前期计划和研究会很有成效. 作为测试人员,您可以执行以下操作:
Your team may or may not need to do any preparation in advance of an iteration. Because priorities change fast in agile development, you don’t want to waste time planning stories that may be postponed. However, if you’re about to implement some new technology, embark on a complex new theme, hope to save time in iteration planning, or your team is divided into different locations, you might find some up-front planning and research to be productive. As a tester, you can do the following:
通过提出问题和获取示例,帮助客户实现“提前清晰”——就每个故事的预期行为达成共识。
Help the customers achieve “advance clarity”—consensus on the desired behavior of each story—by asking questions and getting examples.
积极主动,在迭代之前了解复杂的故事,并确保它们的大小正确。
Be proactive, learn about complex stories in advance of the iteration, and make sure they’re sized correctly.
您并不总是需要提前准备才能在下一次迭代中开始运行。不要做任何不能在迭代期间节省时间或确保更成功地满足客户需求的准备工作。
You don’t always need advance preparation to be able to hit the ground running in the next iteration. Don’t do any preparation that doesn’t save time during the iteration or ensure more success at meeting customer requirements.
协调不同地点之间的关系,促进沟通。有很多工具可以帮助解决这个问题。
Coordinate between different locations and facilitate communication. There are many tools to help with this.
获取示例以帮助说明每个故事。
Obtain examples to help illustrate each story.
在下一次迭代之前为新的和不寻常的功能制定测试策略。
Develop test strategies in advance of the next iteration for new and unusual features.
对现有缺陷进行分类并确定其优先级,以确定是否应为下一次迭代安排任何缺陷。
Triage and prioritize existing defects to determine whether any should be scheduled for the next iteration.
确定是否需要为下一次迭代安排当前不在手边的任何必要测试资源。
Determine whether any necessary testing resources not currently at hand need to be lined up for the next iteration.
敏捷测试人员在迭代计划中扮演着重要的角色,帮助计划测试和开发任务。随着迭代的进行,测试人员积极与客户和开发人员合作,编写有助于指导开发的高级测试,引出和说明示例,确保故事是可测试的。让我们仔细看看敏捷测试人员在每次迭代开始时的活动。
Agile testers play an essential role during iteration planning, helping to plan testing and development tasks. As the iteration gets under way, testers actively collaborate with customers and developers, writing the high-level tests that help guide development, eliciting and illustrating examples, making sure stories are testable. Let’s take a closer look at the agile tester’s activities at the beginning of each iteration.
大多数团队通过计划会议开始他们的新迭代。在此之前可能会进行回顾或“经验教训”会议,以回顾在上一次迭代中哪些工作良好,哪些工作不顺利。尽管回顾的行动项目或“开始、停止、继续”建议会影响即将开始的迭代,但我们将在第 19 章“结束迭代”中将回顾作为迭代结束的活动来讨论。
Most teams kick off their new iteration with a planning session. This might be preceded by a retrospective, or “lessons learned” session, to look back to see what worked well and what didn’t in the previous iteration. Although the retrospective’s action items or “start, stop, continue” suggestions will affect the iteration that’s about to start, we’ll talk about the retrospective as an end-of-iteration activity in Chapter 19, “Wrap Up the Iteration.”
在为迭代计划工作时,开发团队一次讨论一个故事,编写和估算实施所需的所有任务那个故事。如果您提前做了一些工作来为迭代做准备,那么这个计划会议可能会进行得相当快。
While planning the work for the iteration, the development team discusses one story at a time, writing and estimating all of the tasks needed to implement that story. If you’ve done some work ahead of time to prepare for the iteration, this planning session will likely go fairly quickly.
刚接触敏捷开发的团队通常需要大量时间来进行迭代计划会议。Lisa 的团队刚开始时,迭代计划通常需要一整天。现在他们在两三个小时内完成,其中包括回顾的时间。Lisa 的团队使用投影仪来显示他们 wiki 中的用户验收测试用例和满意度条件,以便团队中的每个人都可以看到。他们还投射了他们的在线故事板工具,他们在其中编写任务卡。他们计划会议的另一个传统组成部分是他们轮流提供的一盘点心。图 17-1显示了正在进行的迭代计划会议。
Teams new to agile development often need a lot of time for their iteration planning sessions. Iteration planning often took a whole day when Lisa’s team first started out. Now they are done in two or three hours, which includes time for the retrospective. Lisa’s team uses a projector to display user acceptance test cases and conditions of satisfaction from their wiki so that everyone on the team can see them. They also project their online story board tool, where they write the task cards. Another traditional component of their planning meetings is a plate of treats that they take turns providing. Figure 17-1 shows an iteration planning meeting in progress.
理想情况下,产品负责人和/或其他客户团队成员参与迭代计划、回答问题并提供描述每个故事需求的示例。如果业务方面没有人可以参加,与客户密切合作的团队成员,如分析师和测试人员,可以作为代理人。他们代表客户解释细节并做出决定,或者记下问题以便快速得到答复。如果你的团队在迭代之前与客户一起回顾了故事,你可能认为在迭代计划会议期间你不需要他们在场。但是,我们建议您提供它们以防万一您有其他问题。
Ideally, the product owner and/or other customer team members participate in the iteration planning, answering questions and providing examples describing requirements of each story. If nobody from the business side can attend, team members who work closely with the customers, such as analysts and testers, can serve as proxies. They explain details and make decisions on behalf of the customers, or take note of questions to get answered quickly. If your team went over stories with the customers in advance of the iteration, you may think you don’t need them on hand during the iteration planning session. However, we suggest that they be available just in case you do have extra questions.
正如我们在整本书中所强调的那样,使用示例来帮助团队理解每个故事,并将这些示例转化为驱动编码的测试。按优先顺序处理故事。如果您之前没有与客户讨论过故事,则产品负责人或代表客户团队的其他人首先阅读要计划的每个故事。他们解释故事的目的、它将传递的价值,并举例说明如何使用它。这可能涉及传递示例或在白板上书写。UI 和报告故事可能已经有团队可以研究的线框或模型。
As we’ve emphasized throughout the book, use examples to help the team understand each story, and turn these examples into tests that drive coding. Address stories in priority order. If you haven’t previously gone over stories with the customers, the product owner or other person representing the customer team first reads each story to be planned. They explain the purpose of the story, the value it will deliver, and give examples of how it will be used. This might involve passing around examples or writing on a whiteboard. UI and report stories may already have wire frames or mock-ups that the team can study.
对某些团队有帮助的做法是在迭代计划期间为每个故事一起编写用户验收测试。他们与产品负责人和可能的其他利益相关者一起编写高级测试,当通过时,将表明故事已经完成。这也可以作为迭代“准备工作”的一部分在迭代计划之前不久完成。
A practice that helps some teams is to write user acceptance tests for each story together, during the iteration planning. Along with the product owner and possibly other stakeholders, they write high-level tests that, when passing, will show that the story is done. This could also be done shortly in advance of iteration planning as part of the iteration “prep work.”
故事的大小应该不超过几天就能完成。当我们定期测试小故事时,我们不会立即将它们全部完成并在迭代结束时堆积起来等待测试。如果一个故事已经通过了发布计划和预迭代讨论并且仍然太大,那么这是将它分解成更小的部分的最后机会。即使是一个小故事也可能很复杂。团队可能会通过练习来识别功能的薄片或关键路径。使用示例来指导您,找到最基本的用户场景。
Stories should be sized so they’ll take no more than a few days to complete. When we get small stories to test on a regular basis, we do not have them all finished at once and stacked up at the end of the iteration waiting to be tested. If a story has made it past release planning and pre-iteration discussions and is still too large, this is the final chance to break it up into smaller pieces. Even a small story can be complex. The team may go through an exercise to identify the thin slices or critical path through the functionality. Use examples to guide you, and find the most basic user scenarios.
敏捷测试人员以及其他团队成员对“范围蔓延”保持警惕。当一个故事似乎在四面八方发展时,不要害怕举起红旗。Lisa 的团队有意识地努力指出“bling”或“nice to have”组件,这些组件不是故事功能的核心。这些可以推迟到最后,或推迟,以防故事完成的时间比计划的要长。
Agile testers, along with other team members, are alert to “scope creep.” Don’t be afraid to raise a red flag when a story seems to be growing in all directions. Lisa’s team makes a conscious effort to point out “bling,” or “nice to have” components, which aren’t central to the story’s functionality. Those can be put off until last, or postponed, in case the story takes longer than planned to finish.
作为测试人员,您将尝试将每个故事置于更大系统的上下文中,并评估对其他领域的意外影响的可能性。正如你在发布计划会议上所做的那样,让自己处于不同的心态用户、业务利益相关者、程序员、技术作家以及参与创建和使用该功能的每个人。现在您在详细级别上工作。
As a tester, you’ll try to put each story into the context of the larger system and assess the potential of unanticipated impacts on other areas. As you did in the release planning meeting, put yourself in the different mind-sets of user, business stakeholder, programmer, technical writer, and everyone involved in creating and using the functionality. Now you’re working at a detailed level.
在发布计划章节中,我们使用了这个示例故事:
In the release planning chapter, we used this example story:
作为客户,我想知道根据我选择的送货速度,我的订单需要支付多少运费,所以我可以根据需要进行更改。
As a customer, I want to know how much my order will cost to ship based on the shipping speed I choose, so I can change if I want to.
我们决定采取“薄片”并改变这个故事,假设只有一种运输速度。其他运输速度将在以后的故事。对于这个故事,我们需要根据物品重量和目的地计算运费,我们决定使用 BigExpressShipping 的 API 进行计算。我们的故事现在如图 17-2所示。
We decided to take a “thin slice” and change this story to assume there is only one shipping speed. The other shipping speeds will be later stories. For this story, we need to calculate shipping cost based on item weight and destination, and we decided to use BigExpressShipping’s API for the calculation. Our story is now as shown in Figure 17-2.
团队开始讨论这个故事。
The team starts discussing the story.
测试员: “这个故事是否适用于网站上所有可用的项目?是否有任何物品太重或不符合 5 天送达的条件?
Tester: “Does this story apply for all items available on the site? Are any items too heavy or otherwise disqualified for 5-day delivery?
产品负责人: “我们所有的产品都有 5 天的停产期。隔夜和 2 天的重量不得超过 25 磅。”
Product Owner: “5-day ground is available for all our items. It’s the overnight and 2-day that are restricted to less than 25 lbs.”
测试员: “从业务角度来看,这里的目标是什么?是否可以轻松计算加快结账的成本?你希望鼓励他们检查其他运输方式——那些更有利可图吗?”
Tester: “What’s the goal here, from the business perspective? Making it easy to figure the cost to speed up the checkout? Are you hoping to encourage them to check the other shipping methods—are those more profitable?”
产品负责人: “易用性是我们的主要目标,我们希望结账过程快速,我们希望用户轻松确定订单的总成本,这样他们就不会害怕完成购买。”
Product Owner: “Ease of use is our main goal, we want the checkout process to be quick, and we want the user to easily determine the total cost of the order so they won’t be afraid to complete the purchase.”
程序员: “只要用户输入送货地址,我们就可以默认显示 5 天的送货费用。当我们为其他运输选项制作故事时,我们可以放置按钮以快速弹出这些成本。”
Programmer: “We could have the 5-day shipping cost display as a default as soon as the user enters the shipping address. When we do the stories for the other shipping options, we can put buttons to pop up those costs quickly.”
产品负责人: “这就是我们想要的,预先计算成本。我们将把我们的网站宣传为对客户最友好的网站。”
Product Owner: “That’s what we want, get the costs up front. We’re going to market our site as the most customer friendly.”
测试员: “有什么方法可以让用户搞砸吗?他们会在这个页面上做什么?”
Tester: “Is there any way the user can screw up? What will they do on this page?”
产品负责人: “当我们添加其他运输选项时,他们可以选择更改他们的运输选项。但就目前而言,它真的很简单。我们已经进行了验证,以确保他们的邮政编码与他们为送货地址输入的城市相匹配。”
Product Owner: “When we add the other shipping options, they can opt to change their shipping option. But for now, it’s really straightforward. We already have validation to make sure their postal code matches the city they enter for the shipping address.”
测试员: “如果他们意识到他们弄乱了送货地址怎么办?也许他们不小心给了账单地址。他们怎么能回来更改送货地址?”
Tester: “What if they realize they messed up their shipping address? Maybe they accidentally gave the billing address. How can they get back to change the shipping address?”
程序员: “我们将放置按钮来编辑账单地址和送货地址,这样用户更容易更正错误。我们将在显示运费的页面上显示这两个地址。当我们添加多个送货地址选项时,我们可以稍后扩展它。”
Programmer: “We’ll put buttons to edit billing and shipping addresses, so it will be very easy for the user to correct errors. We’ll show both addresses on this page where the shipping cost displays. We can extend this later when we add the multiple shipping addresses option.”
测试人员: “这将使 UI 易于使用。我知道当我在网上购物时,在订单确认之前无法看到运费让我很烦恼。如果运费贵得离谱而且我不想继续,那我已经浪费了时间。我们希望确保用户不会卡在结账过程中,感到沮丧,然后放弃。因此,他们将看到的下一页是订单确认页面。当用户到达该页面时,运费是否有可能会有所不同?”
Tester: “That would make the UI easy to use. I know when I shop online, it bugs me to not be able to see the shipping cost until the order confirmation. If the shipping is ridiculously expensive and I don’t want to continue, I’ve already wasted time. We want to make sure users can’t get stuck in the checkout process, get frustrated, and just give up. So, the next page they’ll see is the order confirmation page. Is there any chance the shipping cost could be different when the user gets to that page?”
程序员: “不,只要相同的商品仍在购物车中,为我们提供估算成本的 API 就应该始终与实际成本相匹配。”
Programmer: “No, the API that gives us the estimated cost should always match the actual cost, as long as the same items are still in the shopping cart.”
产品负责人: “如果他们更改数量或删除任何项目,我们需要确保立即更改运费以反映这一点。”
Product Owner: “If they change quantities or delete any items, we need to make sure the shipping cost is immediately changed to reflect that.”
正如您在谈话中看到的那样,很多澄清都浮出水面。现在团队中的每个人都对这个故事有了共同的理解。谈论故事的各个方面很重要。以小组形式编写用户验收测试是确保开发团队了解客户需求的好方法。让我们继续监控这个对话。
As you can see by the conversation, a lot of clarification came to light. Everyone on the team now has a common understanding of the story. It’s important to talk about all aspects of the story. Writing user acceptance tests as a group is a good way to make sure the development team understands the customer requirements. Let’s continue monitoring this conversation.
测试人员: “让我们编写一些快速测试以确保我们做对了。”
Tester: “Let’s just write up some quick tests to make sure we get it right.”
客户: “好的,这个例子怎么样?
Customer: “OK, how’s this example?
我可以选择两件有 5 天送货选项的商品,并立即查看我的费用。
I can select two items with a 5-day shipping option and see my costs immediately.
测试员: “很好的开始,但到那时我们还不知道将它运送到哪里。更通用的测试怎么样:
Tester: “Great start, but we won’t know where to ship it to at that point. How about a more generic test like:
确认用户输入送货地址后,默认显示 5 天送货费用。
Verify the 5-day shipping cost displays as the default as soon as the user enters a shipping address.
顾客: “这对我有用。”
Customer: “That works for me.”
基于不同的观点提出问题将有助于澄清故事并让团队做得更好。
Asking questions based on different viewpoints will help to clarify the story and allow the team to do a better job.
当您的团队对故事有了很好的理解后,您就可以开始编写和估算任务卡了。因为敏捷开发通过测试驱动编码,所以我们同时编写测试和开发任务卡。
When your team has a good understanding of a story, you can start writing and estimating task cards. Because agile development drives coding with tests, we write both testing and development task cards at the same time.
如果你做了任何预先计划,你可能已经写好了一些任务卡。如果没有,在迭代计划会议期间写下它们。谁写任务卡并不重要,但团队中的每个人都应该审阅它们并有机会提供他们的意见。我们认识到,在我们开始编码时可能会添加任务,但是在会议期间识别大部分任务并对其进行估算可以让团队很好地了解所涉及的内容。
If you have done any pre-planning, you may have some task cards already written out. If not, write them during the iteration planning meeting. It doesn’t matter who writes the task cards, but everyone on the team should review them and get a chance to give their input. We recognize that tasks may be added as we begin coding, but recognizing most of the tasks and estimating them during the meeting gives the team a good sense of what is involved.
Janet 使用的方法与此类似,但程序员的编码卡停留在“待测试”列中,直到测试任务完成。两张卡片同时移动到“完成”列。
Janet uses an approach similar to this, but the programmer’s coding card stays in the “To Test” column until the testing task has been completed. Both cards move at the same time to the “Done” column.
Story PA-5(图 17-2)的三张测试卡显示了基于重量和目的地的 5 天送达的运费,Lisa 的团队可能会写下这些:
Three test cards for Story PA-5 (Figure 17-2), displaying the shipping cost for 5-day delivery based on weight and destination, that Lisa’s team might write are:
编写 FitNesse 测试以根据重量和目的地计算 5 天的运输成本。
Write FitNesse tests for calculating 5-day ship cost based on weight and destination.
编写用于显示 5 天运送成本的 WebTest 测试。
Write WebTest tests for displaying the 5-day ship cost.
手动测试显示 5 天交货运费。
Manually test displaying the 5-day delivery ship cost.
有些团队更喜欢直接在开发任务卡上写测试任务。这是一个简单的解决方案,因为在测试完成之前任务显然没有“完成”。您正在尝试避免最后完成测试的“迷你瀑布”方法,而程序员觉得她已经完成了,因为她“将故事发送给了 QA”。查看哪种方法最适合您的团队。
Some teams prefer to write testing tasks directly on the development task cards. It’s a simple solution, because the task is obviously not “done” until the testing is finished. You’re trying to avoid a “mini-waterfall” approach where testing is done last, and the programmer feels she is done because she “sent the story to QA.” See what approach works best for your team.
如果故事大量涉及外部各方或共享资源,请编写任务卡以确保不会忘记这些任务,并使估算足够大以允许团队无法控制的依赖性和事件。我们处理运输成本故事的假设团队必须使用托运人的成本计算 API。
If the story heavily involves outside parties or shared resources, write task cards to make sure those tasks aren’t forgotten, and make the estimates generous enough to allow for dependencies and events beyond the team’s control. Our hypothetical team working on the shipping cost story has to work with the shipper’s cost calculation API.
测试员: “有谁知道我们在 BigExpressShipping 与谁合作以获得他们的 API 规范?我们传递给他们什么,只是重量和邮政编码?我们已经可以进行测试了吗?”
Tester: “Does anyone know who we work with at BigExpressShipping to get specs on their API? What do we pass to them, just the weight and postal code? Do we already have access for testing this?”
Scrum Master: “BigExpressShipping 的 Joe 是我们的联系人,他已经发送了这份指定输入和输出格式的文档。他们仍然需要授权从我们的测试系统访问,但这应该在几天内完成。”
Scrum Master: “Joe at BigExpressShipping is our contact, and he’s already sent this document specifying input and output format. They still need to authorize access from our test system, but that should be done in a couple of days.”
测试人员: “哦,很好,我们需要这些信息来编写测试用例。我们将编写一张测试卡来验证我们是否可以访问他们的 API 并收回运费。但是我们怎么知道成本真的是正确的呢?”
Tester: “Oh good, we need that information to write test cases. We’ll write a test card just to verify that we can access their API and get a shipping cost back. But how do we know the cost is really correct?”
Scrum Master: “Joe 为我们提供了一些重量和邮政编码以及预期成本的测试用例,因此我们可以发送这些输入并检查正确的输出。我们还有此电子表格显示一些不同邮政编码的费率。”
Scrum Master: “Joe has provided us with some test cases for weight and postal code and expected cost, so we can send those inputs and check for the correct output. We also have this spreadsheet showing rates for some different postal codes.”
测试人员: “我们应该留出大量时间来确保我们正确访问他们的 API。我打算对这张卡进行高估,以验证使用API 进行测试。也许 API 接口的开发者卡也应该有一个相当保守的估计。”
Tester: “We should allow lots of time for just making sure we’re accessing their API correctly. I’m going to put a high estimate on this card to verify using the API for testing. Maybe the developer card for the interface to the API should have a pretty conservative estimate, too.”
在编写程序员任务卡时,确保编码任务估计包括编写单元测试和程序员进行所有必要测试的时间。用于“端到端”测试的卡片有助于确保从事不同、独立任务的程序员验证所有部分是否协同工作。测试人员应该帮助确保写下所有必要的卡片并且他们有合理的估计。您不希望事后估计,但如果测试估计是编码估计的两倍,则可能值得一谈。
When writing programmer task cards, make sure that coding task estimates include time for writing unit tests and for all necessary testing by programmers. A card for “end-to-end” testing helps make sure that programmers working on different, independent tasks verify that all of the pieces work together. Testers should help make sure all necessary cards are written and that they have reasonable estimates. You don’t want second-guess estimates, but if the testing estimates are twice as high as the coding estimates, it might be worth talking about.
一些团队将测试任务限制在一天或更少的工作时间内,并且懒得在卡片上写下估计的时间。如果一天的工作后任务卡仍然存在,团队会讨论发生这种情况的原因并写新卡以继续前进。这可能会减少开销和记录保存,但如果您将任务输入电子系统,则可能不会。做对您的团队有意义的事情。
Some teams keep testing tasks to a day’s work or less and don’t bother to write estimated hours on the card. If a task card is still around after a day’s work, the team talks about why that happened and writes new cards to go forward. This might cut down on overhead and record-keeping, but if you are entering tasks into your electronic system, it may not. Do what makes sense for your team.
估计错误修复的时间也总是很棘手。如果将现有缺陷作为故事引入,那将非常简单。但是作为迭代的一部分发现的错误呢?
Estimating time for bug fixing is always tricky as well. If existing defects are pulled in as stories, it is pretty simple. But what about the bugs that are found as part of the iteration?
无论您的团队选择估算在迭代期间修复缺陷所花费的时间,无论是包含在故事估算中还是单独跟踪,都要确保始终如一地完成。
However your team chooses to estimate time spent for fixing defects during the iteration, whether it is included in the story estimate or tracked separately, make sure it is done consistently.
估计测试任务时要考虑的另一个项目是测试数据。迭代的开始几乎来不及考虑您需要测试的数据。正如我们在第 15 章“发布或主题规划中的测试人员活动”中提到的,在发布规划期间考虑测试数据,并要求客户帮助识别和获取它。在为下一次迭代做准备时,一定要考虑一下。当迭代开始时,必须创建或获取任何缺少的测试数据,所以不要忘记在估算中考虑到这一点。
Another item to consider when estimating testing tasks is test data. The beginning of an iteration is almost too late to think about the data you need to test with. As we mentioned in Chapter 15, “Tester Activities in Release or Theme Planning,” think about test data during release planning, and ask the customers to help identify and obtain it. Certainly think about it as you prep for the next iteration. When the iteration starts, whatever test data is missing must be created or obtained, so don’t forget to allow for this in estimates.
作为技术团队,我们控制自己的工作量。当我们为每个故事编写任务并将它们发布到我们的(真实的或虚拟的)故事板上时,我们将估计的时间相加或目视检查卡片的数量。我们可以承担多少工作?在 XP 中,我们不能超过上一次迭代中完成的故事点数。在 Scrum 中,我们根据我们认为需要完成的实际时间来承诺一组故事。
We, as the technical team, control our own workload. As we write tasks for each story and post them on our (real or virtual) story board, we add up the estimated hours or visually check the number of cards. How much work can we take on? In XP, we can’t exceed the number of story points we completed in the last iteration. In Scrum, we commit to a set of stories based on the actual time we think we need to complete them.
Lisa 当前的团队在他们的敏捷过程中拥有多年的经验,并且发现他们有时会浪费时间为他们在迭代期间可能没有时间完成的故事编写任务卡。他们从足够多的故事开始,让每个人都忙起来。随着人们开始腾出时间,他们会收集更多故事并计划相关任务。他们可能已经准备好一些故事,以便在完成最初的故事后立即引入。这听起来很容易,但要做到这一点很难,除非您学到了足够多的知识,对故事的规模和团队速度更有信心,并且知道您的团队在给定的时间和特定情况下可以做什么和不能做什么。
Lisa’s current team has several years of experience in their agile process and finds they sometimes waste time writing task cards for stories they may not have time to do during the iteration. They start with enough stories to keep everyone busy. As people start to free up, they pull in more stories and plan the related tasks. They might have some stories ready “on deck” to bring in as soon as they finish the initial ones. This sounds easy, but it is difficult to do until you’ve learned enough to be more confident about story sizes and team velocity, and know what your team can and cannot do in a given amount of time and in specific circumstances.
作为测试人员,您的工作是确保为测试分配足够的时间,并提醒团队测试和质量是整个团队的责任。当团队决定他们可以在迭代中交付多少故事时,问题不是“我们可以完成多少编码?” 但是“我们可以完成多少编码和测试?” 有时故事很容易编码,但测试会非常耗时。作为测试人员,重要的是您只接受尽可能多的故事进入迭代,可以测试。
Your job as tester is to make sure enough time is allocated to testing, and to remind the team that testing and quality are the responsibility of the whole team. When the team decides how many stories they can deliver in the iteration, the question isn’t “How much coding can we finish?” but “How much coding and testing can we complete?” There will be times when a story is easy to code but the testing will be very time consuming. As a tester, it is important that you only accept as many stories into the iteration as can be tested.
如果您必须提交,请谨慎提交。引入另一个故事总是比不得不放弃一个更好。如果您有难以估计的高风险故事,或者某些任务未知或需要更多研究,请为一两个额外的故事写下任务卡,并在场边准备好它们以进行中期迭代。
If you have to commit, commit conservatively. It’s always better to bring in another story than to have to drop one. If you have high-risk stories that are hard to estimate, or some tasks are unknown or need more research, write task cards for an extra story or two and have them ready on the sidelines to bring in mid-iteration.
作为一个团队,我们总是会尽力而为。我们需要记住,只有经过测试,故事才能完成,因此请相应地进行计划。
As a team, we’re always going to do our best. We need to remember that no story is done until it’s tested, so plan accordingly.
当你在看故事时,程序员开始考虑实现,总是想着如何测试它们。一个例子对“测试可测试性”大有帮助。它会对我的测试产生什么影响?第 III 部分,“敏捷测试象限”,给出了很多关于如何设计应用程序以实现有效测试。这是您在编码开始之前考虑故事的可测试性的最后机会。
When you are looking at stories, and the programmers start to think about implementation, always think how you can test them. An example goes a long way toward “testing the testability.” What impact will it have on my testing? Part III, “The Agile Testing Quadrants,” gives a lot of examples of how to design the application to enable effective testing. This is your last opportunity to think about testability of a story before coding begins.
在迭代计划期间,考虑需要测试哪种变体。这可能会引发其他问题。
During iteration planning, think about what kind of variations you will need to test. That may drive other questions.
当可测试性成为问题时,将其作为团队的问题来解决。通过编写测试任务卡开始计划的团队可能在这方面有优势,因为当他们考虑他们的测试任务时,他们会问如何测试这个故事。是否可以在 GUI 后面测试任何功能?是否可以在单元级别进行面向业务的测试?每个敏捷团队都应该首先考虑测试。当您的团队为故事编写开发人员任务卡时,请考虑如何测试故事以及如何对其进行自动化测试。如果程序员还没有编写 TDD 代码或自动化单元测试的习惯,请尝试为每个故事写一张“XUnit”任务卡。为任何需要的测试自动化装置编写编程任务卡。考虑有助于测试的应用程序更改,例如运行时属性和 API。
When testability is an issue, make it the team’s problem to solve. Teams that start their planning by writing test task cards probably have an advantage here, because as they think about their testing tasks, they’ll ask how the story can be tested. Can any functionality be tested behind the GUI? Is it possible to do the business-facing tests at the unit level? Every agile team should be thinking test-first. As your team writes developer task cards for a story, think about how to test the story and how to automate testing for it. If the programmers aren’t yet in the habit of coding TDD or automating unit tests, try writing a “XUnit” task card for each story. Write programming task cards for any test automation fixtures that will be needed. Think about application changes that could help with testing, such as runtime properties and APIs.
如果您因为其他团队正在开发系统的某些部分而遇到类似问题,请编写任务卡与其他团队讨论该问题并提出协调的解决方案。如果无法与其他团队合作,请安排时间集思广益,想出另一个解决方案。至少,要注意这些限制,并相应地调整测试估计并管理相关风险。
If you have similar issues because other teams are developing parts of the system, write a task card to discuss the problem with the other team and come up with a coordinated solution. If working with the other team isn’t an option, budget time to brainstorm another solution. At the very least, be mindful of the limitations, and adjust testing estimates accordingly and manage the associated risk.
当您开始对团队进行一些新事物时,例如新的模板框架或报告库,请记住将其作为风险包含在您的测试计划中。希望您的团队在选择新框架或工具之前考虑了可测试性,并选择了一个可以增强您的测试能力的框架或工具。对所有新事物(包括新领域)的测试任务估计要大方,因为有很多未知数。有时新的领域知识或新技术意味着陡峭的学习曲线。
When you’re embarking on something new to the team, such as a new templating framework or reporting library, remember to include it as a risk in your test plan. Hopefully, your team considered the testability before choosing a new framework or tool, and selected one that enhanced your ability to test. Be generous with your testing task estimates with everything new, including new domains, because there are lots of unknowns. Sometimes new domain knowledge or new technology means a steep learning curve.
与客户或客户代理(如功能分析师)密切合作是我们作为敏捷测试人员最重要的活动之一。当您开始迭代时,您的客户协作也将进入高潮。现在是执行第 8 章“支持团队的面向业务的测试”中描述的所有那些好的活动的时候了。向客户询问示例,询问有关每个故事的功能和行为的开放式问题,围绕白板进行讨论,然后将这些示例转化为测试以驱动编码。
Working closely with customers, or customer proxies such as functional analysts, is one of our most important activities as agile testers. As you kick off the iteration, your customer collaboration will also kick into high gear. This is the time to do all those good activities described in Chapter 8, “Business-Facing Tests that Support the Team.” Ask the customers for examples, ask open-ended questions about each story’s functionality and behavior, have discussions around the whiteboard, and then turn those examples into tests to drive coding.
即使您的产品负责人和/或其他客户在迭代计划之前和期间解释了这些故事,有时在迭代开始时再简单地复习一遍它们也是有帮助的。以前可能不是每个人都听说过,客户可能了解更多。
Even if your product owner and/or other customers explained the stories before and during iteration planning, it’s sometimes helpful to go over them briefly one more time as the iteration starts. Not everyone may have heard it before, and the customer may have more information.
良好的沟通通常需要努力。如果您没有抓住足够的机会提出问题和审查测试用例,请继续并安排定期会议来这样做。如果没有太多要讨论的,会议会进行得很快。在会议中花时间进行有见地的讨论可以节省以后的编码和测试时间,因为您对需求更加确定。
Good communication usually takes work. If you’re not taking enough opportunities to ask questions and review test cases, go ahead and schedule regular meetings to do so. If there’s not much to discuss, the meetings will go quickly. Time in a meeting for an insightful discussion can save coding and testing time later, because you’re more certain of the requirements.
我们想要“大局”测试来帮助程序员在故事的正确方向上开始。像往常一样,我们建议从示例开始并将它们转化为测试。在编码开始之前,您必须进行试验,看看在验收测试级别有多少细节是合适的。Lisa 的团队发现,从示例中提取的高级测试是他们开始故事所需要的。
We want “big picture” tests to help the programmers get started in the right direction on a story. As usual, we recommend starting with examples and turning them into tests. You’ll have to experiment to see how much detail is appropriate at the acceptance test level before coding starts. Lisa’s team has found that high-level tests drawn from examples are what they need to kick off a story.
高级测试应该传达故事背后的主要目的。它们可能包括期望和不期望行为的示例。对于我们之前要求根据订单重量和目的地显示 5 天送达的运费的故事 PA-5(图 17-2),我们的高级测试可能包括:
High-level tests should convey the main purpose behind the story. They may include examples of both desired and undesired behavior. For our earlier Story PA-5 (Figure 17-2) that asks to show the shipping cost for 5-day delivery based on the order’s weight and destination, our high-level tests might include:
验证用户输入送货地址后 5 天的送货费用是否显示为默认值。
Verify that the 5-day shipping cost displays as the default as soon as the user enters a shipping address.
验证估计的运费是否与最终发票上的运费相符。
Verify that the estimated shipping cost matches the shipping cost on the final invoice.
验证用户是否可以单击按钮来更改送货地址,完成后会显示更新的运费。
Verify that the user can click a button to change the shipping address, and when this is done, the updated shipping cost displays.
验证如果用户从购物车中删除项目或将项目添加到购物车,则显示更新的运输选项。
Verify that if the user deletes items from the cart or adds items to the cart, the updated shipping option is displayed.
编写高级测试时,不要局限于 wiki 页面上的文字。例如,如图 15-7所示的测试矩阵可能会工作得更好。有些人使用工作流绘图和图片以图形方式表达测试。Brian Marick [ 2007 ] 有一种绘制图形测试的技术,可以将其转换为 Ruby 测试脚本。模型驱动开发提供了另一种方式来表达故事的高级范围。用例是在“大局”级别表达所需行为的另一种可能途径。
Don’t confine yourself to words on a wiki page when you write high-level tests. For example, a test matrix such as the one shown in Figure 15-7 might work better. Some people express tests graphically, using workflow drawings and pictures. Brian Marick [2007] has a technique to draw graphical tests that can be turned into Ruby test scripts. Model-driven development provides another way to express high-level scope for a story. Use cases are another possible avenue for expressing desired behavior at the “big picture” level.
请参阅参考书目以获取有关图形测试和模型驱动开发的更多信息的链接。
See the bibliography for links to more information on graphical tests and model-driven development.
模型可以快速清晰地传达对 UI 或报告的要求。如果现有报告需要修改,请截取报告的屏幕截图并使用荧光笔、钢笔、铅笔或任何方便的工具。如果您想以电子方式捕获它,请尝试使用 Windows 画图程序或其他图形工具绘制更改并将其发布在描述报告要求的维基页面上。
Mock-ups can convey requirements for a UI or report quickly and clearly. If an existing report needs modifying, take a screenshot of the report and use highlighters, pen, pencil, or whatever tools are handy. If you want to capture it electronically, try the Windows Paint program or other graphical tool to draw the changes and post it on the wiki page that describes the report’s requirements.
See the sample mock-up of UI changes in Chapter 16, “Hit the Ground Running.”
分布式团队需要以电子方式提供的高级测试,而同一地点的团队可以通过白板上的绘图工作得很好,甚至可以让客户坐在他们身边并在他们编码时告诉他们需求。
Distributed teams need high-level tests available electronically, while co-located teams might work well from drawings on a whiteboard, or even from having the customer sit with them and tell them the requirements as they code.
请参阅第 9 章“支持团队的面向业务的测试工具包”,了解有关收集和交流需求的工具的一些想法。
See Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” for some ideas on tools to gather and communicate requirements.
在开始迭代时,重要的是您要快速了解每个故事的基本要求,并以适用于整个团队的方式在上下文中表达它们。我们谈过的大多数敏捷团队都说他们最大的问题是要足够好地理解每个故事,以准确地提供客户想要的东西。他们可能会生成技术上没有错误但与客户所需功能不完全匹配的代码。或者,随着客户澄清需求,他们最终可能会在迭代期间对一个故事进行大量返工,结果没时间完成另一个故事。
What’s important as you begin the iteration is that you quickly learn the basic requirements for each story and express them in context in a way that works for the whole team. Most agile teams we’ve talked to say their biggest problem is to understand each story well enough to deliver exactly what the customer wanted. They might produce code that’s technically bug-free but doesn’t quite match the customer’s desired functionality. Or they may end up doing a lot of rework on one story during the iteration as the customer clarifies requirements, and run out of time to complete another story as a result.
花时间和精力尝试不同的方法,以适合您的领域和环境的方式捕获和表达高级测试。Janet 喜欢说需求是故事 + 对话 + 用户场景或支持图片(如果需要)+ 辅导测试或示例的组合。
Put time and effort into experimenting with different ways to capture and express the high-level tests in a way that fits your domain and environment. Janet likes to say that a requirement is a combination of the story + conversation + a user scenario or supporting picture if needed + a coaching test or example.
请参阅第 8 章,“支持团队的面向业务的测试”,了解更多关于构成需求的内容。
See Chapter 8, “Business-Facing Tests that Support the Team,” for more about what makes up a requirement.
在本章的前面,我们谈到了持续的客户协作的重要性。与客户一起审查高级测试是加强协作和加强沟通的好机会,尤其是对于新的敏捷团队而言。在您的团队习惯于不断讨论故事、需求和测试用例之后,您可能不需要坐下来仔细检查每个测试用例。
Earlier in this chapter we talked about the importance of constant customer collaboration. Reviewing high-level tests with customers is a good opportunity for enforced collaboration and enhanced communication, especially for a new agile team. After your team is in the habit of continually talking about stories, requirements, and test cases, you might not need to sit down and go over every test case.
如果您的团队签订合同开发软件,需求和测试用例可能是您必须提交的正式交付物。即使不是,以客户可以轻松阅读和理解的格式提供测试用例也是一个好主意。
If your team is contracting to develop software, requirements and test cases might be formal deliverables that you have to present. Even if they aren’t, it’s a good idea to provide the test cases in a format that the customers can easily read on their own and understand.
你可以拥有世界上所有的图表和 wiki 页面,但如果没有人看它们,它们将无济于事。直接沟通总是最好的。与程序员坐下来讨论高级测试和需求。一起检查白板图或纸质原型。图 17-4显示了测试人员和程序员通过用户工作流程讨论薄片或线程的图表。如果您与其他地点的团队成员一起工作,请想办法安排电话交谈。如果团队成员难以理解高级测试和需求,您就会知道下次要尝试不同的方法。
You can have all of the diagrams and wiki pages in the world, but if nobody looks at them, they won’t help. Direct communication is always best. Sit down with the programmers and go over the high-level tests and requirements. Go over whiteboard diagrams or paper prototypes together. Figure 17-4 shows a tester and a programmer discussing a diagram of thin slices or threads through a user workflow. If you’re working with a team member in another location, find a way to schedule a phone conversation. If team members have trouble understanding the high-level tests and requirements, you’ll know to try a different approach next time.
具有良好领域知识的程序员可以立即理解一个故事,甚至可以在编写高级测试之前就开始编码。即便如此,与程序员一起从客户和测试人员的角度审阅故事始终是个好主意。他们对故事的理解可能与您的不同,注意不匹配很重要。记住“三的力量”规则,如果有两种意见无法调和,就抓住客户。测试用例还有助于将故事与应用程序的其余部分联系起来。程序员可以使用测试来帮助他们正确编写故事代码。这是您希望尽可能在迭代开始前完成此操作的主要原因——在程序员开始编码之前。
Programmers with good domain knowledge may understand a story right away and be able to start coding even before high-level tests are written. Even so, it’s always a good idea to review the stories from the customer and tester perspective with the programmers. Their understanding of the story might be different than yours, and it’s important to look at mismatches. Remember the “Power of Three” rule and grab a customer if there are two opinions you can’t reconcile. The test cases also help put the story in context with the rest of the application. Programmers can use the tests to help them to code the story correctly. This is the main reason you want to get this done as close to the start of the iteration as you can—before programmers start to code.
第 2 章,“敏捷测试人员的十大原则”,介绍了“三的力量”规则。
Chapter 2, “Ten Principles for Agile Testers,” introduces the “Power of Three” rule.
不要忘记询问程序员他们认为您可能遗漏了什么。代码的高危区域有哪些?他们认为测试应该集中在哪里?获得更多的技术视角将有助于设计详细的测试用例。如果您已经创建了一个测试矩阵,您可能还想再次查看受影响的区域。
Don’t forget to ask the programmers what they think you might have missed. What are the high-risk areas of the code? Where do they think the testing should be focused? Getting more technical perspective will help with designing detailed test cases. If you’ve created a test matrix, you may want to review the impacted areas again as well.
与程序员一起审查测试的一个有益的副作用是发生的交叉学习。作为测试人员,您会接触到他们的想法,并且他们会学习一些他们在其他情况下不会遇到的测试技术。作为程序员,他们可能会更好地理解他们没有考虑过哪些高级测试。
One beneficial side effect of reviewing the tests with the programmers is the cross-learning that happens. You as a tester are exposed to what they are thinking, and they learn some techniques for testing that they would not have otherwise encountered. As programmers, they may get a better understanding of what high-level tests they hadn’t considered.
高级测试用例以及您将在迭代期间编写的可执行测试将构成应用程序文档的核心。在此迭代期间和之后需求会发生变化,因此请确保您的可执行测试用例易于维护。不熟悉敏捷开发的人常常会有没有文档的错误观念。事实上,敏捷项目生成包含可执行测试的可用文档,因此始终是最新的。
High-level test cases, along with the executable tests you’ll write during the iteration, will form the core of your application’s documentation. Requirements will change during and after this iteration, so make sure your executable test cases are easy to maintain. People unfamiliar with agile development often have the misconception that there’s no documentation. In fact, agile projects produce usable documentation that contains executable tests and thus is always up to date.
将可执行测试作为需求文档的一部分的巨大优势在于,很难对它们的结果提出异议。
The great advantage of having executable tests as part of your requirements document is that it’s hard to argue with their results.
组织测试用例和测试并不总是那么简单。许多团队在 wiki 上记录测试和需求。wiki 的灵活性的缺点是您最终可能会遇到混乱的层次结构。您可能无法找到所需的特定要求或示例。
Organizing the test cases and tests isn’t always straightforward. Many teams document tests and requirements on a wiki. The downside to a wiki’s flexibility is that you can end up with a jumble of hierarchies. You might have trouble finding the particular requirement or example you need.
Lisa 的团队定期重新访问其 wiki 文档和 FitNesse 测试,并重构它们的组织方式。如果您在组织需求和测试用例时遇到困难,请花一些时间研究可能有用的新工具。聘请熟练的技术作家是将您宝贵的测试用例和示例放入易于查找的可用信息库中的好方法。
Lisa’s team periodically revisits its wiki documentation and FitNesse tests, and refactors the way they’re organized. If you’re having trouble organizing your requirements and test cases, budget some time to research new tools that might help. Hiring a skilled technical writer is a good way to get your valuable test cases and examples into a usable repository of easy-to-find information.
Chapter 14, “An Agile Test Automation Strategy,” has more on test management.
迭代计划会议为整个迭代设定了基调。在本章中,我们了解了敏捷测试人员如何帮助以良好的开端启动迭代。
The iteration planning session sets the tone for the whole iteration. In this chapter, we looked at what agile testers do to help kick off the iteration to a good start.
在迭代计划期间,测试人员通过提问和考虑所有观点来帮助团队了解故事。
During iteration planning, testers help the team learn about the stories by asking questions and considering all viewpoints.
任务卡需要与开发任务卡一起编写,并根据实际情况进行估算。
Task cards need to be written along with development task cards and estimated realistically.
处理测试任务的另一种方法是将它们直接写在开发人员任务卡上。
Another way of tackling testing tasks is to write them directly on the developer task cards.
团队应该致力于他们可以完成所有测试任务的工作,因为在完全测试之前没有故事完成。
Teams should commit to the work for which they can complete all of the testing tasks, because no story is done until it’s fully tested.
迭代的开始是确保故事可测试并提供足够测试数据的最后机会。
The start of an iteration is the last chance to ensure that the stories are testable and that adequate test data is provided.
测试人员与客户协作,详细探索故事并编写高级测试用例,让程序员开始编码。
Testers collaborate with customers to explore stories in detail and write high-level test cases to let programmers kick off coding.
测试人员与程序员一起审查高级测试和需求,以确保他们沟通良好。
Testers review high-level tests and requirements with programmers to make sure they are communicating well.
测试构成了应用程序文档的核心。
Tests form the core of the application’s documentation.
我们的敏捷测试人员帮助规划发布、适当调整故事大小并确保它们是可测试的。她与客户和开发团队的同事一起,将每个故事的期望行为示例转化为高级用户验收测试。她和她的团队已经准备好了实现商业价值所需的资源和基础设施。现在,团队成员已经拿起任务卡开始写代码了。测试人员下一步做什么,尤其是在任何故事准备好测试之前?
Our agile tester has helped plan the release, size stories appropriately, and make sure they’re testable. She, along with colleagues on the customer and development team, has turned examples of desired behavior for each story into high-level user acceptance tests. She and her team have lined up the resources and infrastructure needed to deliver business value. Now, team members have picked up task cards and started writing code. What do testers do next, especially before any stories are ready to test?
编码的开始是开始编写详细测试的好时机。在迭代前或迭代前几天编写的高级测试,为程序员开始他们自己的测试驱动开发提供了足够的信息。所以现在我们有一些喘息的空间,但如果我们不迅速行动,编码可能会远远领先于测试并朝着错误的方向发展。
The beginning of coding is a good time to start writing detailed tests. The high-level tests written before the iteration, or in the first couple days of it, provide enough information for the programmers to start their own test-driven development. So now we have a bit of breathing room, but if we don’t move quickly, coding could get way ahead of testing and go off in the wrong direction.
现在是时候开始编写可执行的测试来说明故事的细节,以保持开发的顺利进行并帮助测试与编码保持同步。与高级测试一样,我们根据客户提供的示例进行详细测试。
Now’s the time to start writing executable tests that illustrate the details about a story in order to keep development moving forward smoothly and help testing keep pace with coding. Like the high-level tests, we base detailed tests on examples provided by the customers.
在这一点上,我们主要编写将自动化的测试,但我们也在考虑编码完成后需要进行的重要探索性测试。
At this point, we’re mainly writing tests that will be automated, but we’re also thinking ahead to the important exploratory testing we need to do as coding is completed.
作为测试人员,我们很容易被有趣的代码味道和边缘情况分散注意力。然而,如果我们使用测试来指导编码,我们必须从基础开始。编写最简单的快乐路径测试,以证明核心功能有效。
As testers, we’re easily distracted by interesting code smells and edge cases. However, if we’re using tests to guide coding, we have to start with the basics. Write the simplest happy path test you can in order to show that the core functionality works.
为什么可执行测试?我们的工作时间非常紧,程序员和测试人员都没有时间停下来一遍又一遍地运行手动测试。他们确实有时间单击按钮并运行自动化测试。该测试需要以一种使原因尽可能明显的方式失败。理想情况下,我们会将这些测试提供给程序员,以便他们可以在编写代码时执行它们。这就是为什么选择正确的自动化框架如此重要的原因之一。
Why executable tests? We’re working on an extremely tight schedule, and neither the programmers nor the testers have time to stop and run manual tests over and over. They do have time to click a button and run an automated test. That test needs to fail in a way that makes the cause as obvious as possible. Ideally, we would give these tests to the programmers so that they could execute them as they code. That is one reason why picking the right automation framework is so important.
第 14 章“敏捷自动化策略”给出了选择正确工具的指导。
Chapter 14, “An Agile Automation Strategy,” gives pointers for selecting the right tools.
对于某些故事,自动化测试可能需要很长时间。通过保持第一个测试简单,您可以将重点放在设计自动化解决方案上。当简单的测试有效时,值得花时间进行更复杂的测试用例。
For some stories, automating the tests might take a long time. By keeping the first test simple, you keep the focus on designing the automation solution. When the simple test works, it’s worth putting time into more complex test cases.
我们强调自动化的重要性,但 Janet 曾与成功使用清单或电子表格形式的手动测试的团队合作,为程序员提供他们启动所需的信息。然而,为了从长远来看取得成功,这些测试确实需要自动化。
We stress the importance of automation, but Janet has worked with teams that have successfully used manual tests in the form of checklists or spreadsheets to give the programmers the information they need to start. However, to be successful in the long run, these tests do need to be automated.
一旦快乐路径测试起作用,就开始添加更多测试用例。添加边界和边缘条件。测试可能表明程序员误解了需求,或者它们可能表明测试人员误解了需求,或者每个人都没有理解需求的真正含义。重要的是每个人都在谈论它并走上正轨。
As soon as the happy path test works, start adding more test cases. Add boundary and edge conditions. The tests may show that the programmers misunderstood a requirement, or they may show that a tester did, or maybe the requirement’s true meaning eluded everyone. The important thing is that everyone talks about it and gets on track.
当测试人员考虑使用可执行测试验证的新场景时,他们也会考虑手动探索性测试的潜在场景。记下这些以备后用。
As testers think of new scenarios to validate with executable tests, they also think about potential scenarios for manual exploratory testing. Make a note of these for later pursuit.
记住这些测试的目的。他们应该提供示例,告诉程序员要编写什么代码。随着代码的发展,您的测试可以对其提出更多挑战,但要抵制立即将气味跟踪到边缘情况的诱惑。首先让基础知识发挥作用。如果您根据某些风险分析想到更多案例,您以后总是可以添加额外的测试。
Remember the purpose of these tests. They should provide examples that tell the programmers what code to write. As the code evolves, your tests can challenge it more, but resist the temptation to immediately follow smells into edge cases. Get the basics working first. If you think of more cases based on some risk analysis, you can always add extra tests later.
测试人员长期以来一直使用风险分析来帮助确定测试的优先级,并且敏捷开发中已经内置了对风险的考虑。高风险的故事可能会得到更高的规模估计,并且团队在发布和迭代计划期间在优先考虑故事时会考虑风险。
Testers have used risk analysis to help prioritize testing for a long time, and consideration for risk is already built into agile development. High-risk stories may get higher size estimates, and teams consider risk as they prioritize stories during release and iteration planning.
一些快速的风险分析可以帮助您决定首先进行哪些测试以及将精力集中在哪些方面。我们从来没有时间测试所有东西,我们可以使用风险分析来确定多少测试就足够了。
Some quick risk analysis can help you decide what testing to do first and where to focus your efforts. We never have time to test everything, and we can use risk analysis to figure out how much testing is just enough.
如果您的故事非常复杂,您可能希望首先列出与该故事相关的所有潜在风险。这些不限于功能。考虑安全性、性能、可用性和其他“能力”。接下来,对于每个项目,使用 1 到 5 的等级(或适合您的任何等级)对每个项目的业务影响进行评级:1 是低影响,5 是严重的负面影响。
If you have a really complex story, you may want to start by listing all of the potential risks related to the story. These aren’t limited to functionality. Consider security, performance, usability, and other “ilities.” Next, for each item, rate the impact on the business if it were to occur, using a scale of 1 to 5 (or whatever scale works for you): 1 being a low impact, 5 being a critical negative impact.
现在,考虑每个项目发生的可能性,使用相同的尺度:1 表示根本不可能发生,5 表示可能会出现的项目。将两个评级相乘以获得每个项目的总风险评级。这使得您可以轻松地挑选出您的团队应该首先关注其测试工作的领域。低风险项目可以留到最后,或者因为它们的影响很低或极不可能发生,所以可能根本不会解决。
Now, consider the likelihood of each item occurring, using the same scale: 1 for not at all likely to happen, and 5 for items that probably will come up. Multiply the two ratings together to get the total risk rating for each item. This makes it easy to pick out the areas where your team should focus its testing efforts first. Low-risk items can be left for last, or, because their impact is low or they’re highly unlikely to occur, may not be addressed at all.
您的域在这里产生了巨大的变化。如果您正在测试在心脏起搏器中运行的软件,您可能需要在测试中涵盖所有风险,无论它们有多低或不太可能。如果您正在测试供一些受过培训的主题专家使用的内部公司 Web 应用程序,您可以跳过不太可能出现或有明显解决方法的场景。
Your domain makes a huge difference here. If you’re testing software that runs in heart pacemakers, you probably need to cover all risks with your testing no matter how low or unlikely they are. If you’re testing an internal company web application to be used by a few trained subject matter experts, you may be able to skip over scenarios that are unlikely or have an obvious workaround.
考虑图 18-1中的故事。
Consider the story in Figure 18-1.
图 18-2显示了此运输成本案例的可能风险评估。
Figure 18-2 shows a possible risk assessment for this shipping cost story.
项目 8 是风险最高的项目,因此我们希望确保测试更改的送货地址并验证更新后的成本。我们可能想要自动化这个场景的端到端测试。我们不太担心第 5 项;也许我们已经测试了我们的邮政编码验证并且感觉良好,所以我们不需要再测试它。您甚至可能有一个您选择不测试的风险非常低的项目。
Item 8 is the highest-risk item, so we’d want to be sure to test changing shipping addresses and verify the updated costs. We might want to automate an end-to-end test with this scenario. We’re not too worried about item 5; maybe we have already tested our postal code validation and feel good about it, so we don’t need to test it more. You may even have a very low-risk item that you chose not to test.
历史通常是一位好老师。记下过去的问题并确保它们不会再次发生。
History is usually a good teacher. Take note of past issues and make sure they don’t happen again.
在迭代的这一点上,编码和测试继续携手并进。测试人员、程序员、数据库专家和其他团队成员按照示例和测试提供的指南协作开发故事。不同的团队成员可能会贡献他们的特殊专业知识,但他们都觉得有责任确保每个故事都完成。随着工作的进展,他们都了解了这个故事并互相学习。
At this point in the iteration, coding and testing continue hand in hand. Testers, programmers, database experts, and other team members collaborate to develop the stories, following the guidelines provided by examples and tests. Different team members may contribute their particular expertise, but all of them feel responsible for making sure each story is finished. All of them learn about the story and learn from each other as work progresses.
让我们看看团队如何处理图 18-1中的运输成本故事。Patty 程序员拿起一张任务卡来对估计的运费计算进行编码。她已经从早期的讨论中很好地理解了这个故事,但她可能会查看 wiki 页面或故事卡片的背面,测试人员在这些地方写下了一些描述故事目的的叙述,一些故事应该如何运作的例子,以及一些高级测试,以确保她清楚地知道从哪里开始。Tammy Tester 看到编码工作已经开始,并开始编写用于成本计算的 GUI 测试用例。
Let’s look at how a team might work on the shipping cost story in Figure 18-1. Patty Programmer picks up a task card to code the estimated shipping cost calculations. She already understands the story pretty well from earlier discussions, but she may look at the wiki pages or back of the story card where the testers wrote down some narrative describing the purpose of the story, some examples of how it should work, and some high-level tests to make sure she has a good idea of where to start. Tammy Tester sees that coding work has begun and starts to write behind-the-GUI test cases for the cost calculations.
该团队在计划期间同意首先根据送货地址和物品重量计算 5 天的送货成本。项目只能在北美大陆内运送,但验证将在表示层完成,因此成本计算测试可以假设仅考虑有效目的地作为输入。他们正在使用运输合作伙伴提供的成本计算 API,Tammy 询问 Patty 在哪里可以找到算法,以便她可以自己计算成本以编写测试。Tammy 在他们的 behind-the-GUI 测试工具中编写了她能想到的最简单的测试用例。我们在图 18-3中将其显示为一个简单的表格。
The team had agreed during planning to start by calculating the 5-day shipping cost based on the shipping address and item weight. Items can only be shipped within continental North America, but that validation will be done in the presentation layer, so the cost calculation tests can assume only valid destinations are considered for input. They’re using a cost calculation API provided by the shipping partner, and Tammy asks Patty where to find the algorithms so she can figure the cost herself in order to write the tests. Tammy writes the simplest test case she can think of in their behind-the-GUI test tool. We show it as a simple table in Figure 18-3.
Patty 尚未完成使该测试通过的代码,因此 Tammy 开始着手该故事的另一项测试任务,设置测试环境以与运输合作伙伴的测试系统一起工作。
Patty hasn’t finished the code that would make this test pass yet, so Tammy starts working on another testing task for the story, setting up the test environment to work with the shipping partner’s test system.
因为这个故事和测试非常简单,所以 Patty 和 Tammy 没有像讨论更复杂的故事那样讨论测试设计和调整它。他们也不需要向产品负责人询问更多问题。Patty 打电话给 Tammy,让她知道这个简单的测试现在可以正常工作了。Tammy 编写了更多测试用例,在美国尝试不同的权重和目的地。这些都很好。她尝试了加拿大的邮政编码,但测试出现异常。她向帕蒂展示了这一点,帕蒂意识到 API 默认为美国邮政编码,并且需要加拿大和墨西哥代码的国家/地区代码。她还没有为其他国家编写任何单元测试。他们修改测试输入,Patty 与 Paul Programmer 结对更改调用 API 的代码。现在测试看起来像图 18-4。
Because this story and the test are so straightforward, Patty and Tammy don’t discuss the test design and tweak it as they might on more complex stories. They also haven’t needed to ask the product owner more questions yet. Patty calls Tammy over to show her that the simple test is now working. Tammy writes up more test cases, trying different weights and destinations within the United States. Those all work fine. She tries a Canadian postal code, and the test gets an exception. She shows this to Patty, who realizes that the API defaults to U.S. postal codes, and requires a country code for codes in Canada and Mexico. She hadn’t written any unit tests yet for other countries. They revise the test inputs, and Patty pairs with Paul Programmer to change the code that calls the API. Now the test looks something like Figure 18-4.
这个简单的例子说明了编码和测试之间的反复来回。不同的团队采取不同的方法。Patty 和 Tammy 可能会在编码和测试方面结对。Tammy 可能会与 Paul 合作编写 fixture 来自动化测试。Tammy 可能在远程办公室,使用在线协作工具与 Patty 一起工作。Patty 可能会自己编写可执行的故事测试,然后编写代码使它们工作,实践真实的故事测试驱动开发。关键是测试和编码是所有团队成员参与的一个开发过程的一部分。
This simple example illustrates the iterative back-and-forth between coding and testing. Different teams take different approaches. Patty and Tammy might pair on both the coding and testing. Tammy might pair with Paul to write the fixture to automate the test. Tammy might be in a remote office, using an online collaboration tool to work with Patty. Patty might write the executable story tests herself and then write the code to make them work, practicing true story test-driven development. The point is that testing and coding are part of one development process in which all team members participate.
Tammy 可以继续识别新的测试用例,包括边缘用例和边界条件,直到她觉得所有风险区域都已被最少数量和种类的测试用例覆盖。她可能会用网站上最重的商品发送到最昂贵的目的地进行测试。她可能会尝试大量购买相同的物品。一些极端情况可能不太可能发生,以至于她不去理会它们,或者她决定运行测试,但在测试通过后没有将其包含在回归套件中。某些测试最好在 UI 可用后手动完成。
Tammy can continue to identify new test cases, including edge cases and boundary conditions, until she feels all risk areas have been covered by the minimum amount and variety of test cases. She might test with the heaviest item available on the website sent to the most expensive destination. She might test having a large quantity of the same item. Some edge cases may be so unlikely that she doesn’t bother with them, or she decides to run a test but after it passes doesn’t include it in the regression suite. Some tests might be better done manually after a UI is available.
Patty 编写了将夏威夷作为运输目的地的单元测试,但 Tammy 认为只有大陆目的地是可以接受的。他们都不确定军事邮政信箱目的地是否可以接受。他们去见 Polly 产品负责人,询问她的想法。他们正在使用三的力量。当出现分歧或问题时,拥有三个不同的观点是确保您获得良好解决方案并且以后不必重新讨论问题的有效方法。如果参与讨论的人中有一个不熟悉这个话题,其他人就必须整理思路,把它解释清楚,这总是有帮助的。让不同角色的人参与进来有助于确保对需求的更改不会悄悄发生,并在以后让团队成员感到惊讶。
Patty has written unit tests with Hawaii as the shipping destination, but Tammy believes that only continental destinations are acceptable. Neither of them is sure whether military post office box destinations are acceptable. They go see Polly Product-Owner to ask what she thinks. They’re using the Power of Three. When disagreements or questions arise, having three different viewpoints is an effective way to make sure you get a good solution and you won’t have to rehash the issue later. If one of the participants in the discussion isn’t familiar with the topic, the others will have to organize their thoughts to explain it clearly, which is always helpful. Involving people in different roles helps make sure that changes to requirements don’t fly under the radar and surprise team members later.
当意想不到的问题出现时,三的力量规则是一个很好的起点。根据问题的严重性或复杂性,您可能需要召集更多人,甚至是整个团队。如果运输合作伙伴的 API 被证明太慢以至于网站上的响应时间无法接受怎么办?开发团队和客户团队都需要快速探索替代解决方案。
When unexpected problems arise, as they always do, the Power of Three rule is a great place to start. You may need to pull in more people, or even the whole team, depending on the severity or complexity of the issue. What if the shipping partner’s API proves to be so slow that the response time on the website will be unacceptable? Both the development team and the customer team need to quickly explore alternative solutions.
Paul 正在寻找要处理的编程任务。虽然预估运费故事的 UI 任务仍在任务板上的“待办事项”栏中,但他对从购物车中删除商品的故事更感兴趣,因此他拿起其中一张卡片。没有人有时间开始为那个故事编写可执行测试,所以他自己一头扎了进去。
Paul looks for a programming task to work on. Although the UI tasks for the estimated shipping cost story are still in the “to do” column on the task board, he’s more interested in the story to delete items out of the shopping cart, so he picks up one of those cards. Nobody has time to start writing the executable tests for that story, so he plunges ahead on his own.
现在团队有两个故事。他们真的不知道完成这两个故事需要多少时间。一个更好的方法是让 Paul 开始处理第一个故事的 UI 任务,这样故事就可以更快地完成。当一个故事完成时(意味着所有代码都已编写和测试),您确切地知道还有多少工作要做:零。如果灾难来袭并且没有其他故事在本次迭代中完成,则至少有一个完整的故事要发布。
Now the team has two stories going. They don’t really know how much time it will take to finish either story. A much better approach would be for Paul to start working on a UI task for the first story so that story can be finished sooner. When a story’s done (meaning all of the code is written and tested), you know exactly how much work is left to do on it: zero. If disaster struck and no other stories got finished this iteration, there is at least one completed story to release.
完成整个故事不是测试概念,而是测试人员应该提倡和遵循的概念。如果程序员已经开始编写故事,请确保有人也开始为该故事进行测试任务。这是一种平衡行为。如果没有人为删除项目故事编写过高级测试怎么办?也许这是最高的测试优先级?通常,完成一个故事应该是团队可以继续下一个故事之前的目标。
Completing the whole story isn’t a testing concept, but it’s one that testers should promote and follow. If a programmer has started coding on a story, make sure someone has also started working on testing tasks for that story. This is a balancing act. What if nobody has written even high-level tests for the delete items story? Maybe that’s the highest testing priority? Usually, finishing a story should be the goal before the team can move on to the next story.
除非团队非常小,否则在任何给定时间总是有不止一个故事在进行中。这可能会更困难,但请尝试一次专注于完成一个故事。Patty 即将结束关于运费的故事,而 Paul 已经开始讨论删除项目的故事。帕蒂遇到了障碍,她不知道该如何解决。Paul 帮助她完成代码,以便 Tammy 可以完成她的探索性测试,他们可以将故事标记为“完成”。现在他们可以更好地了解完成本次迭代还有多少工作要做(或者至少,还有多少工作是他们不需要处理的)。
Unless the team is very small, there is always more than one story in progress at any given time. It might be more difficult, but try to focus on finishing one story at a time. Patty is about to wrap up the shipping cost story, and Paul has moved on to the delete items story. Patty runs into a snag, and she isn’t sure how to solve it. Paul helps her to finish the code so that Tammy can finish her exploratory testing and they can mark the story “done.” Now they have a better idea of how much they have left to finish this iteration (or at least, how much they don’t still have to work on).
有时,如果程序员和测试人员结对一起完成每个故事,则可以同时完成几个不同的故事。如果故事很小且独立,则此方法有效。您不希望看到的是程序员在没有同时完成测试任务的情况下开始编码。
Sometimes, several different stories can be done at the same time if a programmer and tester pair up to complete each story together. This works if the stories are small and independent. What you don’t want to see is programmers starting coding without testing tasks being completed at the same time.
一旦可测试的代码块可用,并且指导其编码的自动化测试通过,就需要时间更深入地探索功能。尝试不同的场景并了解有关代码行为的更多信息。您应该有用于批评产品的测试的任务卡,包括面向业务和技术的。在完成所有这些类型的测试之前,故事还没有“完成”。
As soon as testable chunks of code are available, and the automated tests that guided their coding pass, take time to explore the functionality more deeply. Try different scenarios and learn more about the code’s behavior. You should have task cards for tests that critique the product, both business- and technology-facing. The story’s not “done” until all of these types of tests are complete.
当故事的所有任务(测试除外)都已完成时,这一点变得更加重要。现在您应该能够从故事线索的一端到另一端进行测试,并测试其间的所有变化。不要推迟这个测试。您可能会发现需求在故事中,但在推动开发的测试中遗漏了,因此在代码中遗漏了。现在是编写那些缺失的测试和代码的时候了。在团队仍然专注于故事的同时,填补所有空白并增加更多价值。以后这样做会花费更多。
This becomes more important when all tasks except testing are complete for a story. Now you should be able to test from one end of the story’s thread to the other end, with all of the variations in between. Don’t put this testing off. You may find requirements that were in the story but were missed with the tests that drove development and are thus missing in the code. Now’s the time to write those missing tests and code. Fill in all of the gaps and add more value while the team is still focused on the story. Doing this later will cost much more.
请注意,您在测试最终故事中学到的一些东西可能被认为是“拥有的很好”,可能使功能更易于使用或更快,这些项目不是原始故事的一部分。咨询您的客户。如果有时间在迭代中添加它,并且业务可以使用额外的价值,那就继续吧。现在添加这些添加物要便宜得多。但是不要因为花太多时间添加投资回报率不高的“金光闪闪”而危及其他故事。
Be aware that some of what you learn in testing the final story may be considered “nice to have,” perhaps making the functionality easier to use or faster, items that weren’t part of the original story. Consult with your customer. If there’s time to add it in the iteration, and the business can use the extra value, go ahead. These additions are much cheaper to add now. But don’t jeopardize other stories by spending too much time adding “bling” that doesn’t have a big ROI.
第 10 章“面向业务的产品评判测试”和第 11 章“面向技术的评测产品的测试”将帮助您确保涵盖评判产品的所有必要测试。
Chapter 10, “Business-Facing Tests that Critique the Product,” and Chapter 11, “Tecnology-Facing Tests that Critique the Product,” will help you make sure you cover all of the necessary tests that critique the product.
如果您的探索性测试让团队和客户意识到故事未涵盖重要功能,请为未来的迭代编写新故事。严格控制“范围蔓延”,否则您的团队将没有时间交付您最初计划的价值。
If your exploratory testing leads the team and the customers to realize that significant functionality wasn’t covered by the stories, write new stories for future iterations. Keep a tight rein on “scope creep” or your team won’t have time to deliver the value you planned originally.
评价产品的面向技术的测试通常最好在编码期间完成。现在是了解设计是否无法扩展或是否存在安全漏洞的时候了。
Technology-facing tests to critique the product are often best done during coding. This is the time to know if the design doesn’t scale, or if there are security holes.
我们的小插图描述了一个团队编写和使用详细的测试来驱动编码,展示了测试人员和程序员的密切协作。随着编码和测试的进行,这种情况会继续。一起工作可以增强团队交付正确产品的能力,并提供许多传授技能的机会。程序员学习新的测试方法,他们会在编写代码时更好地测试自己的代码。测试人员更多地了解编码过程以及正确的测试如何使编码过程变得更容易。
Our vignette describing a team writing and using detailed tests to drive coding shows how closely testers and programmers collaborate. This continues as coding and testing proceed. Working together enhances the team’s ability to deliver the right product and provides many opportunities to transfer skills. Programmers learn new ways of testing, and they’ll be better at testing their own code as they write it. Testers learn more about the process of coding and how the right tests might make it easier.
Paul Programmer 已经完成了估计运输选项故事的用户界面,但他还没有签入。他请 Tammy 过来和他坐在一起,并演示最终用户如何在结帐过程中输入送货地址。估计的运费立即显示。Tammy 更改了送货地址并看到新的费用出现。她输入的邮政编码与地址的其余部分不匹配,并看到相应的错误消息出现。UI 对他们俩来说都不错,因此 Paul 签入了代码,而 Tammy 继续对其进行探索性手动测试。
Paul Programmer has completed the user interface for the estimated shipping options story, but he hasn’t checked it in yet. He asks Tammy to come sit with him and demonstrates how the end user would enter the shipping address during the checkout process. The estimated shipping cost displays right away. Tammy changes the shipping address and sees the new cost appear. She enters a postal code that doesn’t match the rest of the address and sees the appropriate error message appear. The UI looks good to both of them, so Paul checks in the code, and Tammy continues with her exploratory manual testing of it.
珍妮特喜欢让程序员在这些结对测试期间“开车”,同时她会观察发生的情况。她发现这比在程序员注视下控制键盘和鼠标要有效得多。
Janet likes to have the programmer “drive” during these pair testing sessions while she watches what happens. She finds that it is far more effective than taking control of the keyboard and mouse while the programmer watches.
Tammy 特别关心更改送货地址和重新计算估计成本,因为他们将其确定为风险区域。她发现,如果她显示估算费用,然后转到账单地址页面,然后回来更改送货地址,则估算费用不会正确更改。她让保罗来观察这种行为。他意识到会话缓存存在问题并返回修复它。
Tammy is especially concerned with changing the shipping address and having the estimated cost recalculate, because they identified that as a risky area. She finds that if she displays the estimated cost, goes ahead to the billing address page, and then comes back to change the shipping address, the estimated costs don’t change properly. She gets Paul to come observe this behavior. He realizes there is a problem with session caching and goes back to fix it.
向某人展示问题并一起解决它比在缺陷跟踪系统中提交错误并等待某人有时间查看它要有效得多。如果团队不在同一地点,就更难做到。如果团队成员在截然不同的时区工作,这就更难了。坚持最直接的沟通方式。丽莎的一位队友所在的时区比现在早 12.5 小时。他工作到深夜,在需要时,他会打电话给丽莎,他们一起研究测试结果和示例。
Showing someone a problem and working through it together is much more effective than filing a bug in a defect tracking system and waiting for someone to have time to look at it. It’s harder to do if the team isn’t co-located. If team members are working in vastly different time zones, it’s even harder. Stick to the most direct communication available to you. One of Lisa’s teammates is in a time zone 12½ hours ahead. He works late into his nighttime, and when needed, he calls Lisa and they work through test results and examples together.
向另一个人展示 GUI 的简单行为可能会帮助 Paul 意识到他实施了一些错误的行为。同样,如果 Tammy 在使她的 GUI 测试脚本工作时遇到问题,解释这个问题可能足以让她意识到是什么原因造成的。如果没有人可以查看您刚刚编写的代码或帮助您调试问题,有时向您自己大声解释它会有所帮助。“橡皮鸭”和“大声思考”是解决你自己问题的非常有效的方法。珍妮特喜欢把她自己的小橡皮鸭放在她的桌子上,以提醒自己在提问之前要三思。
The simple act of showing the GUI to another person may help Paul realize he’s implemented some erroneous behavior. Similarly, if Tammy is having trouble getting her GUI test script to work, explaining the problem might be enough for her to realize what’s causing it. If there is nobody available to look at what you’ve just coded or help you debug a problem, it sometimes helps to explain it out loud to yourself. “Rubber Ducking” and “Thinking Out Loud” are surprisingly effective ways to solve your own problems. Janet likes to have her own little rubber duck sitting on her desk to remind herself to think before she asks.
对于开发团队成员来说,埋头编造故事而忘记让客户了解情况是非常容易的。除了在遇到问题时咨询业务专家外,我们还需要向他们展示我们目前已交付的成果。
It’s shockingly easy for development team members to get their heads down cranking out stories and forget to keep customers in the loop. In addition to consulting business experts when we have questions, we need to show them what we’ve delivered so far.
希望在编码开始之前,您能够与客户或可以代表客户的人一起审查测试用例。如果没有,永远不会太晚。对于客户需要更多地参与可执行测试细节的情况,一定要找到适合他们和技术团队成员的测试工具。
Hopefully, you were able to review test cases with customers, or with someone who could represent the customer, before coding began. If not, it’s never too late. For situations where customers need to be more involved with the details of the executable tests, be sure to find test tools that work for them as well as for technical team members.
正如我们在上两章中所描述的,您可能已经与客户讨论过实体模型或纸质原型。如果模拟报告或界面的任务保留在迭代计划中,请记住保持过程简单。例如,不要编写 HTML 原型,而在白板上绘图也可以。我们希望流程尽可能简单;简单是一个核心价值。
As we described in the last two chapters, you may have already gone over mock-ups or paper prototypes with your customers. If tasks to mock up a report or interface remain in the iteration plan, remember to keep the process simple. For example, don’t code an HTML prototype when drawing on a whiteboard will do just as well. We want to keep the process as simple as possible; simplicity is a core value.
一旦编码的用户界面或报告准备就绪,即使它仍然很初级,缺少所有功能或显示硬编码数据,也要将其展示给适当的客户。没有人能提前准确解释他们想要什么。他们需要通过查看、感受和使用应用程序来了解它是否正确。你可能不是能够在迭代中期实施大的改变,但如果你早点开始,可能会有时间进行小的调整,你的客户就会知道会发生什么。
As soon as a coded user interface or report is ready, even if it’s still rudimentary, lacking all features or displaying hard-coded data, show it to the appropriate customers. Nobody can explain exactly what they want ahead of time. They need to see, feel, and use the application to know if it’s right. You may not be able to implement big changes mid-iteration, but if you start early, there may be time for minor tweaks, and your customers will know what to expect.
迭代评审会议是一个很好的机会来展示团队交付了什么并获得下一次迭代的反馈,但不要等到那时才从客户那里获得输入。让他们参与整个迭代。
The iteration review meeting is a great opportunity to show what the team delivered and get feedback for the next iteration, but don’t wait until then to get input from customers. Keep them involved throughout the iteration.
虽然我们陷入了快速迭代的节奏,但我们也需要停下来花时间更好地了解业务。花一些时间与业务人员讨论他们的工作以及哪些方面可以通过新软件功能得到增强。您越了解客户的业务,就越能提供优质的产品。
Although we get caught up in the fast pace of iterations, we also need to stop and take time to understand the business better. Spend some time talking to business people about their jobs and what aspects might be enhanced with new software features. The better you understand your customer’s business, the better you can be at providing a good product.
有些团队实际上长期与业务人员坐在一起,以便他们每天都参与实际业务。
Some teams actually sit with the business people permanently so that they are involved with the actual business on a daily basis.
敏捷测试人员是主动的。我们不会坐等工作来找我们。习惯于瀑布流程的测试人员可能会觉得在故事 100% 完成之前无事可做。在敏捷迭代期间,这很少是真的。与程序员一起工作,以便他们尽早生成一些可测试的代码。前面介绍的运输成本算法就是一个很好的例子。它可以完全独立地进行测试,无需访问数据库或用户界面。或者,可以在访问真实数据的服务完成之前用硬编码数据去除用户界面,并且可以自行测试表示层的行为。
Agile testers are proactive. We don’t sit and wait for work to come to us. Testers who are accustomed to a waterfall process may feel there’s nothing to do until a story is 100% complete. That’s rarely true during an agile iteration. Work with programmers so that they produce some testable piece of code early on. The shipping cost algorithm presented earlier is a good example. It can be tested completely in isolation, without needing to access the database or the user interface. Alternatively, the user interface could be stubbed out with hard-coded data before the services accessing the real data are complete, and the behavior of the presentation layer can be tested by itself.
我们知道许多团队都在为如何跟踪错误或是否要跟踪错误的问题而苦苦挣扎。正如 Tom 和 Mary Poppendieck 在他们的书《实施精益软件开发:从概念到现金》[2006] 中所写,缺陷队列是返工队列,因此是浪费。有些团队一发现错误就简单地修复它们。他们编写一个单元测试来重现错误,修复代码以使测试通过,签入测试和错误修复,然后继续。如果稍后有人破坏了那段代码,测试将捕获回归。
We’ve known many teams that struggle with the question of how to track bugs, or whether to track them at all. As Tom and Mary Poppendieck write in their book Implementing Lean Software Development: From Concept to Cash [2006], defect queues are queues of rework and thus collection points for waste. Some teams simply fix bugs as soon as they’re discovered. They write a unit test to reproduce the bug, fix the code so the test passes, check in the test and the bug fix, and go on. If someone breaks that piece of code later, the test will catch the regression.
其他团队发现在缺陷跟踪系统 (DTS) 中记录问题和修复很有价值,尤其是在代码发布后才发现的问题。他们甚至可能会在生产中的错误中寻找模式,并进行根本原因分析以了解如何防止类似问题再次发生。尽管如此,缺陷系统并没有提供一个关于如何生成更高质量代码的面对面交流的良好论坛。
Other teams find value in documenting problems and fixes in a defect tracking system (DTS), especially problems that weren’t caught until after code was released. They may even look for patterns in the bugs that got to production and do root cause analysis to learn how to prevent similar issues from recurring. Still, defect systems don’t provide a good forum for face-to-face communication about how to produce higher-quality code.
第 5 章,“转换典型流程”,讨论您的团队可能想要或不想使用缺陷跟踪系统的原因。
Chapter 5, “Transitioning Typical Processes,” talks about why your team may or may not want to use a Defect Tracking System.
Lisa 和她的测试伙伴们更愿意在发现问题后立即与程序员交谈。如果程序员可以立即修复它,则无需在任何地方记录错误。如果没有程序员可以立即处理该问题,并且有可能忘记该错误,他们会为它写一张卡片或将其输入到他们的 DTS 中。
Lisa and her fellow testers prefer to talk to a programmer as soon as a problem is found. If the programmer can fix it immediately, there’s no need to log the bug anywhere. If no programmer is available immediately to work on the problem, and there’s a possibility the bug might be forgotten, they write a card for it or enter it into their DTS.
我们将此部分添加到本章,因为这是您遇到问题的时候。您一直在先编写测试,但在与程序员合作时发现了问题。你记录错误吗?如果是这样,如何?您一直在进行探索性测试,并从标记为完成的故事中发现了一个错误。您是否为此记录了错误?让我们更多地讨论缺陷并考虑对您和您的团队开放的选项。
We’ve added this section to this chapter because this is when you run into the problem. You have been writing tests first, but are finding problems as you work with the programmer. Do you log a bug? If so, how? You’ve been doing your exploratory testing and found a bug from a story that was marked done. Do you log a bug for that? Let’s discuss more about defects and consider options that are open to you and your team.
首先,让我们谈谈缺陷与特性。软件开发中的一个古老问题是,“什么是错误”?我们听到的一些答案是:偏离要求或行为不符合预期。当然,也有一些非常明显的缺陷,例如不正确的输出或不正确的错误消息。但真正重要的是用户对产品质量的看法。如果客户说这是一个缺陷,那么它就是一个缺陷。
First, let’s talk about defects versus features. The age-old question in software development is, “What is a bug”? Some answers we’ve heard are: It’s a deviation from the requirements or it’s behavior that is not what was expected. Of course, there are some really obvious defects such as incorrect output or incorrect error messages. But what really matters is the user’s perception of the quality of the product. If the customer says it is a defect, then it is a defect.
在敏捷中,我们有机会与客户合作,让他们满意地解决问题。客户不必事先考虑所有可能的功能和细节。当他们看到某些东西时改变主意对他们来说是可以的。
In agile, we have the opportunity to work with customers to get things fixed to their satisfaction. Customers don’t have to try to think of every possible feature and detail up front. It is okay for them to change their minds when they see something.
最后,如果需要修复它是错误还是功能真的很重要吗?客户选择优先事项和价值主张。如果软件质量对客户来说比获得所有新功能更重要,那么我们应该尝试修复我们发现的所有缺陷。
In the end, does it really matter if it is a bug or a feature if it needs to be fixed? The customer chooses priorities and the value proposition. If software quality is a higher priority for the customer than getting all of the new features, then we should try to fix all defects as we find them.
团队中的客户利用他们的知识在日常开发中为团队提供最好的建议。然而,当产品进入 UAT 并暴露给更大的客户群时,总会有以错误或新增强形式出现的请求。
Customers on the team use their knowledge to give the best advice they can to the team on day-to-day development. However, when a product goes to UAT and is exposed to a larger customer base, there will always be requests in the form of bugs or new enhancements.
考虑缺陷的一种方式是技术债务。缺陷在系统中停留且未被发现的时间越长,影响就越大。确实,让错误在代码库中恶化会对代码质量、系统直观性、系统灵活性、团队士气和速度产生负面影响。修复错误代码中的一个缺陷可能会揭示更多,因此维护任务需要更长的时间。
One way of thinking about defects is as technical debt. The longer a defect stays in the system and goes undetected, the greater the impact. It also is true that leaving bugs festering in a code base has a negative effect on code quality, system intuitiveness, system flexibility, team morale, and velocity. Fixing one defect in buggy code may reveal more, so maintenance tasks take longer.
第 6 章,“测试的目的”,解释了测试如何帮助管理技术债务。
Chapter 6, “The Purpose of Testing,” explains how tests help manage technical debt.
Janet 鼓励与她合作的团队努力实现对错误数量的“零容忍”。新的敏捷团队通常很难相信它可以完成。在 Janet 工作的一个组织中,她向五个项目团队中的每一个提出挑战,看看他们在每次迭代结束时如何接近零未解决的错误,并在发布时达到零错误。
Janet encourages teams that she works with to strive for “zero tolerance” toward bug counts. New agile teams usually have a hard time believing it can be done. In one organization Janet was working with, she challenged each of the five project teams to see how close they could come to zero bugs outstanding at the end of each iteration, and zero at release time.
作为任何开发的一部分,您总是需要做出权衡。您的团队可能决定发布带有一些突出错误的版本,因为人们认为推出新功能比修复低级错误更重要。
As part of any development, you will always need to make trade-offs. Your team may decide to release with some outstanding bugs because it is deemed more important to get new functionality out the door than to fix low-level bugs.
团队已经解决了如何以多种不同方式处理缺陷的问题。一些团队将所有错误都放在任务卡上。其他团队选择写一张卡片,对其进行估算,并将其安排为一个故事。还有一些人建议为每个错误添加一个测试——这样您就不必记录缺陷,只需记录测试。
Teams have solved the problem of how to handle defects in many different ways. Some teams put all of their bugs on task cards. Other teams have chosen to write a card, estimate it, and schedule it as a story. Still others suggest adding a test for every bug—that way you don’t have to record the defect, just the test.
有一种正确的方法吗?当然不是!但是,您怎么知道什么适合您的团队?我们有一些建议可以帮助您选择和决定什么是适合您的。想想你的团队和你的产品,以及什么可能适合你的情况。首先,我们将讨论我们应该记录哪些缺陷,然后我们将讨论何时应该修复它们,最后我们将看看选择什么媒体。正确的组合将取决于您的团队在敏捷之旅中走了多远以及您的产品有多成熟。
Is there one right way? Of course not! But, how do you know what is right for your team? We have some suggestions to help you choose and decide what is right for you. Think about your team and your product and what might work in your situation. First, we’ll talk about what defects we should log, then we’ll talk a bit about when you should fix them, and finally we’ll look at what media to choose. The right combination will depend on how far along your team is in its agile journey and how mature your product is.
并非所有错误都需要记录,但团队经常纠结哪些应该记录,哪些不需要记录。我们建议您尽可能避免创建缺陷报告。首先与真人交谈,只有在确实是需要更改产品或程序员无法立即解决的真正问题时才生成缺陷报告。
Not all bugs need to be logged, but teams often struggle with which ones should be recorded and which ones don’t need to be. We recommend that you avoid creating a defect report if possible. Have a conversation with a real person first, and only produce a defect report if it is truly a real problem that demands a change to the product or the programmers just can’t get to it right away.
不要记录单元测试失败。如果您所在的团队正在实践 TDD(测试驱动开发)并且对其单元测试有很好的覆盖率,那么您就会知道构建期间失败的测试不应该被记录下来。在持续集成构建期间失败的测试是程序员立即解决问题的信号。记录这些错误是多余的,而且是浪费时间。
Don’t log unit test failures. If you are part of a team that is practicing TDD (test-driven development) and has good coverage with its unit tests, you know that failed tests during the build should not be logged. A failed test during the continuous integration build is a signal for the programmers to address the problem right away. Logging these bugs would be redundant and a waste of time.
许多团队都有在单元级别之上运行回归测试的构建,例如 GUI 后面的测试和通过 GUI 进行的测试。当其中一个构建失败时,您是否应该在 DTS 中记录错误?
Many teams have builds that run regression tests above the unit level, such as tests behind the GUI and tests through the GUI. When one of these builds fails, should you log the bug in a DTS?
失败的测试本身就是一种记录错误。但有时,就像 Lisa 的情况一样,需要添加更多信息才能进行有效且干净的修复,因此记录缺陷是有必要的。
Failing tests in themselves are a type of recorded bug. But sometimes, as in Lisa’s case, more information needs to be added to allow for an effective and clean fix, so logging the defect is warranted.
不要记录可以立即修复的错误,特别是如果您要以其他方式将它们记录在电子 DTS 中。如果您的团队与程序员,并在故事完成后立即进行配对测试,我们强烈建议您不要记录这些错误,只要程序员立即解决它们即可。当您注意到问题时,请与程序员讨论并确定它们是否是真正的问题。如果需要,请与客户交谈,但做一些笔记,以便记住您所看到的内容,以便在需要时调整您的测试。
Don’t log bugs that can be fixed immediately, especially if you would otherwise record them in an electronic DTS. If your team is working closely with the programmers and is practicing pair testing as soon as a story is completed, we strongly recommend that you don’t log those bugs as long as the programmer addresses them right away. As you notice issues, talk them over with the programmer and decide whether they are real issues or not. Talk to the customer if you need to, but make a couple of notes so you remember what you saw so you can adjust your tests if needed.
如果您使用索引卡来记录错误,您可能需要将索引卡放在任务板上(或电子板上的卡片)以作为提醒。
If you are using index cards to log bugs, you may want to put an index card up on the task board (or a card on your electronic board) just as a reminder.
记录无法立即修复的错误。我们尽早强调测试,以便在程序员仍在处理故事时捕获尽可能多的错误。我们知道在早期发现时修复它们会更便宜;但是,有时我们只是没有立即抓住它们。程序员已经转向另一个故事,现在不能放下一切来修复它。这些是日志记录的良好候选者。有时,“错误”实际上是一个遗漏的需求,需要作为一个故事来处理——为未来的迭代进行估计和优先级排序。
Do log bugs that can’t be fixed right away. We stress testing early in order to catch as many bugs as possible while the programmers are still working on the story. We know it is cheaper to fix them when caught early; however, sometimes we just don’t catch them right away. The programmer has moved on to another story and can’t drop everything to fix it now. Those are the ones that are good candidates for logging. Sometimes a “bug” is really a missed requirement and needs to be handled as a story—estimated and prioritized for a future iteration.
记录遗留系统中发生的错误。如果您的产品已经存在很长时间,它可能有许多潜伏在后台等待被发现的错误。当您找到它们时,您有几个选择。如果你的产品负责人认为修复它们是值得的,那么记录这些错误,它们可以作为产品待办事项列表的一部分进行优先排序。但是,如果它们已经存在了很长时间并且没有引起任何问题,您的产品负责人可能会认为不值得修复它们。在这种情况下,不要费心记录它们。他们无论如何都不会得到解决,所以不要浪费你的时间。
Do log bugs that occur in the legacy system. If your product has been around a long time, it likely has a number of bugs that have been lurking in the background just waiting to be discovered. When you find them, you have a couple of choices. If your product owner thinks it is worthwhile to fix them, then log the bugs and they can be prioritized as part of the product backlog. However, if they have been around a long time and cause no issues, your product owner may decide it is not worth fixing them. In this case, don’t bother logging them. They will never get addressed anyhow, so don’t waste your time.
记录所有生产错误。当您的应用程序投入生产时,应记录客户发现的所有错误。根据它们的严重程度,这些错误可能会在下一个版本发布时立即修复,或者将对它们进行估计、确定优先级并放入您的产品待办事项列表中。
Do log all production bugs. When your application is in production, all bugs found by the customer should be logged. Depending on their severity, these bugs may be fixed immediately, at the time of the next release, or they’ll be estimated, prioritized, and put in your product backlog.
共有三个选项。您发现的所有错误都需要进行分类,以确定您是现在修复它们、稍后修复它们,还是根本不修复它们。这种分类可能就像与程序员讨论以确定它们是否真的一样简单他正在研究的故事中的错误。分类可能是与产品所有者的讨论,以确定下一次迭代是否应该有另一个故事。分类也可能是与客户确定优先修复哪些错误的正式过程。
There are three options. All bugs you find need to be triaged to determine if you fix them now, fix them later, or don’t fix them at all. This triage may be as simple as a discussion with the programmer to determine if they are really bugs in the story he is working on. The triage may be a discussion with the product owner to determine if there should be another story for the next iteration. The triage may also be a formal process with the customers to prioritize which bugs to fix.
您可以立即修复的错误越多,您的应用程序产生的技术债务就越少,您拥有的“缺陷”库存就越少。越早发现缺陷,修复它们的成本也就越低。在iSixSigma Magazine的一篇文章中,Mukesh Soni [2008] 引用了 IBM 的一份报告,指出修复产品发布后发现的错误的成本是设计期间发现的错误的四到五倍,比识别的错误高出 100 倍在维护阶段(见图18-5)。
The more bugs you can fix immediately, the less technical debt your application generates and the less “defect” inventory you have. Defects are also cheaper to fix the sooner they are discovered. In an article in iSixSigma Magazine, Mukesh Soni [2008] quotes a report from IBM that the cost to fix an error found after product release was four to five times as much as one uncovered during design, and up to 100 times more than one identified in the maintenance phase (see Figure 18-5).
图 18-5显示了基于分阶段方法的统计数据,但该统计数据仍然适用于敏捷开发。修复在开发过程中发现的错误比在开发之后更便宜。
Figure 18-5 shows a statistic based on phased methodology, but the statistic still holds true for agile development. It is cheaper to fix bugs that are found during development than after.
如果在开发新功能时发现缺陷,或者是另一个错误修复的副作用,则应该自动修复。但是,像往常一样,这要谨慎应用。例如,如果发现程序员说很难修复并且可能会破坏产品稳定性的错误,则应将其带给客户以确定优先级。
If a defect is found while developing a new feature, or is a side effect from another bug fix, it should be automatically fixed. But, as usual, this is to be applied with prudence. For example, if a bug is found that the programmers say will be difficult to fix and may destabilize the product, it should be taken to the customers to prioritize.
如果您在开发过程中修复错误,则可以减少该过程后期出现的错误。您的团队速度可以包括修复错误的时间。随着时间的推移,您的团队成员会很好地了解他们花了多长时间来修复测试人员为故事发现的错误。希望很少。如果你的团队是一个新的敏捷团队,可能会有相当多的错误逃避开发,但随着团队对工具和流程越来越熟悉,发现的错误数量将会减少。首先,尝试对一个故事进行估算,以包括两个小时或半天的时间来修复相关的错误。
If you fix the bugs during development, you lessen the presence of bugs later in the process. Your team velocity can include time to fix bugs. Over time, your team members will get a good idea of how long they spend on fixing bugs found by the testers for a story. Hopefully, there are few. If your team is a new agile team, there may be quite a few bugs that escape development, but as the team gets more comfortable with the tools and the processes, the number of bugs found will lessen. To start, try making the estimate for a story to include two hours or half a day for fixing associated bugs.
不同的团队有不同的处理缺陷的方法。一些团队认为,所有发现的缺陷在被列入修复清单之前应该由客户确定优先级。他们认为完全由客户来决定它们是否真的是缺陷,如果是,是否应该修复它们。
Different teams have different ways of handling defects. Some teams believe that all defects found should be prioritized by the customers before they get put on the list to fix. They believe it is completely up to the customer to determine whether they really are defects, and if so, whether they should be fixed.
您的团队已经识别出一个缺陷,但知道它不会得到修复。也许那部分代码以后需要完全重写,因为功能会发生变化,或者它可能只是一个低优先级的问题或太模糊以至于您的客户可能永远找不到它。无法修复的原因有很多。如果您的分类确定是这种情况,我们建议您关闭该错误。不要假装有一天你会修好它。
Your team has recognized a defect, but know it won’t get fixed. Perhaps that section of code needs a complete rewrite later because the functionality will change, or perhaps it is just such a low-priority issue or so obscure that your customers may never find it. There are a multitude of reasons why it won’t get fixed. If your triage determines this is the case, we suggest you just close the bug. Don’t keep it open pretending that you will fix it someday.
当我们谈论媒体时,我们指的是记录错误的各种方式。它可以是缺陷跟踪系统或索引卡,或者您可能选择根本没有物理记录。
When we talk about media, we mean the variety of ways you can log a bug. It could be a defect tracking system or index cards, or maybe you choose to have no physical record at all.
索引卡(无论是在线计划和跟踪系统中的真实卡还是虚拟卡)不会为大量文书细节留出太多空间,但是当它们固定在故事板上时,它们确实可以很好地显示未决问题,尤其是如果它们是另一种颜色。一些团队使用屏幕打印并将它们钉在卡片背面或将详细信息写在文本文件中,甚至在手持录音机上以音频形式记录步骤。
Index cards (whether real or virtual cards in an online planning and tracking system) don’t leave a lot of room for a lot of clerical details, but they do give great visibility to outstanding issues when they are pinned on the story board, especially if they are in another color. Some teams use screen prints and staple them to the back of the card or write the details in a text file, or even record steps in audio form on a hand-held voice recorder.
有很多选项,但我们建议您选择一个包含足够信息的选项,以指导某人重现问题或在程序员准备修复问题时集中讨论。卡片是有形的。DTS 中的五百个错误只是一个数字。一叠 500 张卡片令人印象深刻。
There are lots of options, but we would suggest that you pick one that contains enough information to guide someone to reproduce a problem or to focus a discussion when the programmer is ready to fix it. The card is tangible. Five hundred bugs in a DTS are just a number. A stack of 500 cards is impressive.
Use cards in the following circumstances:
您是一个纪律严明的敏捷团队,并且在迭代中修复所有错误。
You are a disciplined agile team and are fixing all bugs within an iteration.
您想让团队看到错误。
You want to make bugs visible to the team.
没有什么能阻止您同时拥有索引卡和 DTS。
There is nothing stopping you from having both index cards and a DTS.
在以下情况下使用 DTS:
Use a DTS in the following circumstances:
你的团队是分布式的。
Your team is distributed.
您需要出于审计目的跟踪错误或在发行说明中捕获它们。
You need to track bugs for audit purposes or to capture them in release notes.
您有逃脱迭代的错误,您需要记住稍后修复它们。
You have bugs that escape an iteration and you need to remember to fix them later.
您的遗留系统存在大量缺陷。
You have a legacy system with a large number of defects.
不管怎样,您可能希望使用某种 DTS 来记录某些错误。这并不意味着您需要将它们全部记录下来。明智地选择记录哪些内容。
One way or the other, you will likely want to have some kind of DTS to log some of the bugs. This does not mean you need to log them all. Be smart about which ones you do log.
你为什么不记录错误?与我们合作过的大多数团队都为自己设定了规则,即没有单元测试就无法修复错误。如果您还有一个功能性自动化套件,那么您可以用它们来捕获更大的错误。论点是,如果有一个测试可以捕获错误,则无需记录错误。从修复错误中学到的任何东西都被记录在测试和代码中。但是,您需要认识到并非所有测试都易于自动化。
Why wouldn’t you log a bug? Most teams that we have worked with have set rules for themselves that no bug is fixed without a unit test. If you also have a functional automation suite, then you can catch the larger bugs with those. The argument is that if there is a test that will catch the bug, you have no need to log the bug. Anything learned from fixing the bug was captured in the test and the code. However, you need to recognize that not all tests are easy to automate.
在以下情况下使用测试捕获错误:
Use tests to capture bugs in the following circumstance:
您的团队纪律严明,并为发现的每个错误编写测试。
Your team is disciplined and writes tests for every bug found.
随着团队的成熟,他们会找到适合他们的程序。他们消除了多余的任务。他们更加熟练地使用故事卡、故事董事会和项目积压。他们有效地使用测试,并了解要记录哪些错误以及哪些指标对他们的团队有意义。在本节中,我们将分享其他团队为他们找到的一些想法。
As teams mature, they find procedures that work for them. They eliminate redundant tasks. They become more practiced at using story cards, story boards, and project backlogs. They use tests effectively, and learn which bugs to log and what metrics make sense to their team. In this section, we’ll share some ideas that other teams have found work for them.
设定规则,例如“粉红色卡片(错误)的数量在任何时候都不应超过十张。” 每次进行团队回顾时都要重新审视这些。如果您的缺陷率正在下降,请不要担心。如果趋势相反,请花时间分析错误的根本原因并创建新规则来缓解这些错误。
Set rules like, “The number of pink cards (bugs) should never get higher than ten at any one time.” Revisit these each time you have a team retrospective. If your defect rate is going down, no worries. If the trend is the opposite, spend time analyzing the root cause of bugs and create new rules to mitigate those.
不要忘记修复在迭代过程中发现的低优先级错误,因为它们会影响未来的开发。根据我们的经验,“低优先级”和“快速修复”之间似乎存在很强的相关性,尽管我们没有确凿的事实来支持这一点。我们建议在小的、孤立的错误变成大的、纠结的错误之前阻止它们。
Don’t forget to fix low-priority bugs found during the iteration as well, because they have an effect on future development. In our experience, there seems to be a strong correlation between “low priority” and “quick to fix,” although we don’t have hard facts to support that. We suggest stopping small, isolated bugs before they become large, tangled bugs.
如果您在一个区域中发现了很多错误,请考虑将它们组合成一个增强功能或故事。
If you find a lot of bugs in one area, think about combining them into an enhancement or story.
如果一个“错误”确实是错过的功能,选择为错误写一张卡片并将其安排为一个故事。就像任何其他故事一样,对这些故事进行估计和优先级排序。请注意,错误故事可能不会像产品待办事项列表中的新用户故事那样受到关注。创建故事、确定优先级和安排时间也需要时间。
If a “bug” is really missed functionality, choose to write a card for the bug and schedule it as a story. These stories are estimated and prioritized just like any other story. Be aware that bug stories may not receive as much attention as the new user stories in the product backlog. It also takes time to create the story, prioritize, and schedule it.
如果错误只是简单地记录在 DTS 中,那么重要信息可能会从项目中丢失。当我们编写验收测试来驱动开发时,我们倾向于关注所需的行为。从缺陷中了解不良行为并将其转化为故事是生成正确功能的重要补充。
If bugs are simply logged in a DTS, important information might be effectively lost from the project. When we write acceptance tests to drive development, we tend to focus on desired behavior. Learning about undesired behavior from a defect, and turning that into stories is a vital addition to producing the right functionality.
每个团队都需要确定适合自己的流程,以及如何使该流程易于查看。下面的故事是关于一个对珍妮特有效的过程。
Each team needs to determine the process that works for it, and how to make that process easily visible. The following story is about one process that worked for Janet.
这种方法对这个团队很有效,因为团队中有很多纪律,而且大多数新错误如果是新功能或更改功能的一部分,就会在迭代中得到修复。唯一进入积压工作的错误是被认为是低风险的遗留错误。
This approach worked for this team because there was a lot of discipline in the team, and most new bugs were fixed in the iteration if they were part of the new or changed functionality. The only bugs that went into the backlog were legacy bugs that were deemed low risk.
我们建议使用尽可能简单的系统,并根据需要应用复杂性。根据我们的经验,测试优先生成的代码在签入时基本上没有错误。如果您在新代码中发现很多错误,您的团队需要找出原因并采取行动。尽量缩短编码、集成和测试的周期,让程序员得到有关代码质量的即时反馈。也许遗留代码的某些错误部分需要重新设计,以免让您的团队陷入技术债务。也许您需要与业务专家更紧密地合作以了解所需的功能。
We suggest using as simple a system as possible and applying complexity as required. Code produced test-first is, in our experience, fairly free of bugs by the time it’s checked in. If you’re finding a lot of bugs in new code, your team needs to figure out why, and take action. Try to shorten the cycle of coding, integrating and testing so that programmers get immediate feedback about code quality. Perhaps some buggy section of legacy code needs to be redesigned before it mires your team in technical debt. Maybe you need to work more closely with the business experts to understand the desired functionality.
另一个想法可能是创建一个持续的“开始、停止、继续”列表,以便您可以在迭代回顾期间记住一些问题。
Another idea might be to create an ongoing “start, stop, continue” list so that you can remember some of the issues during the iteration retrospective.
More on retrospectives in Chapter 19, “Wrap Up the Iteration.”
每日站会帮助团队保持他们需要的密切沟通。团队中的每个人都了解任务和故事的当前状态,并且可以互相帮助克服障碍。通常,听到程序员描述他们正在处理的任务会提供一个线索,表明他们可能误解了客户的需求。这表明需要在站立后进行小组讨论。如果测试人员需要帮助解决出现的测试问题,她可能会要求团队在站会后留下来讨论。遗漏的任务通常会在站立时被识别出来,并且可以当场写新卡片。
The daily stand-up helps teams maintain the close communication they need. Everyone on the team learns the current status of tasks and stories, and can help each other with obstacles. Often, hearing programmers describe tasks they’re working on provides a clue that they may have misunderstood the customer’s requirements. That signals the need for a group discussion after the stand-up. If a tester needs help with a testing issue that’s come up, she might ask the team to stay after the stand-up to talk about it. Missed tasks are often identified during stand-ups, and new cards can be written on the spot.
站会是查看进展情况的好时机。使用大的、可见的图表,例如故事板、燃尽图和其他视觉提示,以帮助保持专注并了解您的状态。如果迭代的末尾临近,并且故事的编码似乎“停滞不前”,请举起红旗并询问团队可以做些什么。也许一些配对或额外的帮助会让事情顺利进行。Lisa 经常注意到还有很多测试要做而时间不多了。她请求帮助收拾残局。整个团队专注于完成每个故事需要做什么,并讨论最佳方法。
The stand-up is a good time to look at progress. Use big, visible charts such as story boards, burndown charts, and other visual cues to help keep focus and know your status. If the end of the iteration is drawing near, and coding on a story seems “stuck,” raise a red flag and ask the team what can be done about it. Perhaps some pairing or extra help will get things going. Lisa has often noted when there’s a lot of testing left to do and time is running out. She asks for help to pick up the slack. The whole team focuses on what needs to be done to complete each story and talks about the best approach.
当团队使用电子媒体来跟踪故事时,往往会忘记故事板。珍妮特发现同时拥有这两者似乎是重复工作,但团队取得进展的可见性远远超过编写任务卡并在完成时移动它们的额外开销。拥有故事板可以让您的团队在站立会议期间或在您与团队外的人谈论您的进度时集中注意力。
When teams use an electronic medium for keeping track of stories, there is a tendency to forget the story board. Janet finds that having both may seem like a duplication of effort, but the visibility of progress to the team far outweighs the extra overhead of writing up the task cards and moving them as they are completed. Having the story board gives your team focus during the stand-ups or when you are talking to someone outside the team about your progress.
测试人员可以通过帮助确保每个人都进行足够的沟通来帮助保持迭代的顺利进行。当程序员开始编写故事时与他们交谈,并确保他们理解它。Lisa 发现她可以在团队 wiki 上编写她想要的所有测试和示例,但如果没有人愿意阅读它们,它们就无济于事。如有疑问,她会与拿起任务卡的程序员一起检查需求和测试。
Testers can help keep the iteration progressing smoothly by helping make sure everyone is communicating enough. Talk to programmers when they start working on a story, and make sure they understand it. Lisa finds that she can write all of the tests and examples she wants on the team wiki, but if nobody bothers to read them, they don’t help. When in doubt, she goes over requirements and tests with the programmer who picks up the task cards.
程序员在开发故事时总会有问题,即使他们非常了解业务和故事。如果有客户最好回答问题,因为那是最直接的沟通。测试人员不应该妨碍它;然而,我们观察到业务专家有时难以解释需求,或者程序员只是理解错误,无法与客户达成共识。三的力量适用于此。测试人员可以帮助客户和程序员找到共同语言。
Programmers will always have questions as they develop a story, even if they understand the business and the story well. It’s best if a customer is available to answer questions, because that is the most direct communication. Testers shouldn’t get in the way of that; however, we’ve observed that business experts sometimes have trouble explaining a requirement, or a programmer simply gets the wrong idea and can’t get on the same page with the customer. The Power of Three applies here. Testers can help customers and programmers find a common language.
促进沟通通常涉及在白板上绘图、模拟界面、列出可能受影响的其他区域或通过实际示例进行操作。每当沟通似乎进入死胡同,或混乱猖獗时,要求一个新的例子并专注于此。
Facilitating communication usually involves drawing on a whiteboard, mocking up interfaces, listing other areas that might be affected, or working through real examples. Whenever communication appears to reach a dead end, or confusion is rampant, ask for a new example and focus on that.
根据需要完成尽可能多的示例,直到团队充分了解系统的不同方面。如果不起作用,请尝试其他格式。例如,如果画在白板上的图片不足以理解故事,请尝试使用电子表格或业务专家熟悉的其他格式。
Work through as many examples as you need until the team understands enough different aspects of the system. Try a different format if it’s not working. For example, if pictures drawn on the whiteboard aren’t sufficient to understand the story, try spreadsheets or some other format that’s familiar to the business experts.
正如我们在其他章节中提到的,团队成员位于不同地点和不同时区意味着您必须更加努力地进行沟通。电话、电子邮件和即时消息构成了通信的基础,但更好的协作工具一直在开发。
As we’ve noted in other chapters, having team members in different locations and different time zones means you have to work harder at communication. Phones, email, and instant messaging form the basics of communication, but better collaboration tools are developed all the time.
第 9 章,“支持团队的面向业务的测试工具包”,讨论了一些可以帮助分布式团队的工具
Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” talks about some tools that can help distributed teams
您将需要进行试验,看看什么对您的分布式团队有效。使用回顾来评估协作和沟通是否需要改进,并集思广益以改进方法。作为一名测试人员,您可能在帮助流程改进项目方面拥有丰富的经验。只需考虑将改善沟通作为持续改进需求之一。
You will need to experiment to see what works for your distributed team. Use retrospectives to evaluate whether collaboration and communication need improving, and brainstorm ways to improve. You, as a tester, may have a lot of experience in helping with process improvement projects. Just think about improving communication as one of those continual improvement needs.
为了我们的项目取得成功,我们都需要能够相互良好沟通。当团队位于不同的地理位置时,他们可能需要加倍努力才能保持联系。
We all need to be able to communicate well with each other for our projects to succeed. When teams are in diverse geographic locations, they might have to work twice as hard to stay in constant touch.
除非你在一个刚刚开始其自动化工作的团队中,否则你有覆盖以前迭代故事的自动化回归测试。希望这些作为持续构建过程的一部分运行,或者至少作为日常构建过程的一部分运行。如果他们不是,请让您的团队将实施这个关键基础设施作为优先事项,并与他们一起集思广益如何做到这一点。计划下一次迭代的时间以开始构建过程。
Unless you’re on a team that’s just starting its automation efforts, you have automated regression tests covering stories from previous iterations. Hopefully, these are running as part of a continual build process, or at least part of a daily build process. If they aren’t, ask your team to make implementing this critical infrastructure a priority, and brainstorm with them how this might be done. Plan time in the next iteration to start a build process.
程序员应该在签入新代码之前运行所有自动化单元测试。但是,单元测试可能会在持续构建中失败,要么是因为有人在签入之前忘记运行它们,要么是因为运行时环境或 IDE 不同。我们进行单元测试是有原因的,所以每当一个失败时,团队的最高优先级(除了 showstopper 生产问题)应该是修复它并让构建再次运行。
Programmers should run all automated unit tests before checking in new code. However, unit tests may fail in the continual build, either because someone forgot to run them before check-in, or because of a difference in runtime environment or IDE. We have unit tests for a reason, so whenever one fails, the team’s highest priority (apart from a showstopper production issue) should be to fix it and get the build working again.
团队采用不同的方法来确保他们的构建保持“绿色”。Lisa 的团队有一个构建过程,每次构建后都会通过电子邮件发送结果。如果构建失败,签入失败的人通常会立即修复它。如果不清楚构建失败的原因,团队成员将聚在一起进行调查。他们的 ScrumMaster 有一个毛绒玩具,她把它放在“破坏构建”的人的桌子上,作为视觉提醒,它必须立即修复。
Teams take different approaches to make sure their build stays “green.” Lisa’s team has a build process that emails results after every build. If the build fails, the person who checked in the failure usually fixes it right away. If it’s not clear why the build failed, team members will get together to investigate. Their ScrumMaster has a stuffed toy that she puts on the desk of the person who “broke the build,” as a visual reminder that it has to be fixed right away.
一些团队使用红绿灯、环境球体、GUI 构建监控工具或其他电子视觉方式来显示构建状态。当灯变红时,就该停止新开发并修复构建了。另一种技术是在每个人的 IDE 中弹出一个屏幕,显示构建失败,并且弹出窗口不会消失,直到您单击“Ok,我会修复构建”。享受它的乐趣,但保持构建运行是一项严肃的工作。
Some teams use a traffic light, ambient orb, GUI build monitoring tool, or other electronic visual way to show the build status. When the lights turn red, it’s time to stop new development and fix the build. Another technique is to have a screen pop up in everyone’s IDE showing that the build has failed, and the popup won’t go away until you click “Ok, I’ll fix the build.” Have some fun with it, but keeping the build running is serious business.
在极端情况下,您可能不得不暂时注释掉一个失败的测试,直到它可以被诊断出来,但这是一种危险的做法,尤其是对于新手团队而言。如有必要,团队中的每个人都应该停止他们正在做的事情,直到构建再次运行。
In extreme cases, you may have to temporarily comment out a failing test until it can be diagnosed, but this is a dangerous practice, especially for a novice team. Everyone on the team should stop what they’re doing if necessary until the build works again.
构建需要提供即时反馈,因此请保持简短。如果构建花费的时间超过代码签入的平均频率,构建就会开始堆积,测试人员无法获得他们需要测试的代码。XP 的构建时间指南是十分钟 [ Fowler, 2006 ]。Lisa 的团队试图将构建时间控制在八分钟以内,因为他们经常检查。
The build needs to provide immediate feedback, so keep it short. If the build takes longer than the average frequency of code check-ins, builds start to stack up, and testers can’t get the code they need to test. The XP guideline for build time is ten minutes [Fowler, 2006]. Lisa’s team tries to keep the build less than eight minutes, because they check in so often.
花费时间过长的测试,例如更新数据库的测试、单元级别以上的功能测试或 GUI 测试脚本,应该在单独的构建过程中运行。如果团队的硬件有限,他们可能不得不在晚上运行带有全套测试的“完整”构建,以及在工作时间持续运行只有单元测试的“持续”构建。拥有一个单独的、持续的“完整”构建以及所有回归测试套件是值得投资的。Lisa 的团队每 90 分钟从他们的“完整”构建中获得反馈,这已经事实证明,它在解决回归问题方面具有无可估量的价值。这个辅助测试套件不会阻止程序员检查他们的代码。
Tests that take too long, such as tests that update the database, functional tests above the unit level, or GUI test scripts, should run in a separate build process. If the team is limited in hardware, they might have to run the “full” build with the full suite of tests at night and the “ongoing” build that has only unit tests continually during working hours. Having a separate, continual “full” build with all of the regression test suites is worth the investment. Lisa’s team gets feedback every 90 minutes from their “full” build, and this has proven invaluable in heading off regression issues. This secondary suite of tests does not stop a programmer from checking in their code.
在迭代期间,您正在自动化新测试。一旦这些通过,就将它们适当地添加到回归套件中。您可能不需要回归套件中包含的每个边缘案例或排列,并且您希望保持回归套件足够快以提供及时的反馈。随着每个故事的完成,确认其功能的测试应该包含在回归套件中,并成为常规构建周期的一部分。
During the iteration, you’re automating new tests. As soon as these pass, add them to the regression suite, as appropriate. You may not need every edge case or permutation included in the regression suite, and you want to keep the regression suites fast enough to provide timely feedback. As each story is completed, tests that confirm its functionality should be included in the regression suite and be part of the regular build cycle.
回归测试本身必须处于某种形式的版本控制之下。最好将它们保存在与生产代码相同的源代码控制系统中。这样,当您为生产发布标记代码时,该标记还包含与该代码一起使用的所有测试版本。至少,每天备份测试代码。
The regression tests themselves must be under some form of version control. It’s best to keep them in the same source code control system as the production code. That way, when you tag the code for production release, the tag also contains all of the versions of the tests that worked with the code. At minimum, keep a daily backup of the test code.
当测试被添加到回归套件中时,它们的目的就会改变。它们不再存在以帮助推动开发,并且预计它们不会发现新的错误。生活中的唯一目的是检测系统中意想不到的变化或副作用。
When tests have been added to the regression suite, their purpose changes. They no longer exist to help drive development, and they are not expected to find new bugs. There sole purpose in life is to detect unexpected changes or side effects in the system.
希望您编写了任务卡来在更大的应用程序的上下文中测试故事,并对系统的其他部分进行回归测试,以确保新故事没有产生负面影响。您可能已经自动化了其中一些端到端测试,例如第 12 章“测试象限总结”中的示例。
Hopefully, you wrote task cards to test the story in the context of the larger application and regression test other parts of the system to ensure the new story hasn’t had a negative effect. You may have automated some of those end-to-end tests like the example in Chapter 12, “Summary of Testing Quadrants.”
但有时,即使您有一大套回归测试,手动探索性测试也是合适的。在您也完成这些任务之前,故事还没有“完成”。
But sometimes, even if you have a large suite of regression tests, manual exploratory testing can be appropriate. The story isn’t “done” until you’ve completed these tasks as well.
当您开始迭代时,请确保测试环境、测试数据和测试工具已到位以适应测试此迭代的故事。希望您已经预料到这些需求,但有些需求可能只有在您开始编写故事时才会变得明显。与数据库专家、系统管理员和其他团队成员协作,以设置所需的任何其他基础设施。
As you start the iteration, make sure that test environments, test data, and test tools are in place to accommodate testing this iteration’s stories. Hopefully you’ve anticipated these needs, but some requirements might only become obvious when you start working on a story. Collaborate with database experts, system administrators, and other team members to set up any additional infrastructure needed.
您可能为此迭代引入了外部资源,以帮助进行性能、可用性、安全性或其他形式的测试。根据需要让他们参与与客户的站会和讨论。与他们配对并帮助他们了解团队的目标。这是学习新技能的机会。
You may have brought in outside resources for this iteration to help with performance, usability, security, or other forms of testing. Include them in stand-ups and discussions with the customers as needed. Pair with them and help them understand the team’s objectives. This is an opportunity to pick up new skills.
在第 5 章“转换典型流程”中,我们略微谈到了度量的目的,但由于度量对于了解您的编码和测试活动的进展情况至关重要,因此我们将在此处更深入地研究它们。在开始测量数据点并开始分析结果的所有工作之前,了解您要解决的问题。在本节中,我们将介绍团队通过迭代收集的一些典型度量。
In Chapter 5, “Transitioning Typical Processes,” we talked a bit about the purpose of metrics, but because metrics are critical to understanding how your coding and testing activities are progressing, we’ll delve into them more here. Know what problem you are trying to solve before you start measuring data points and going to all the work of analyzing the results. In this section, we’ll cover some of the typical measurements that teams gather through the iteration.
第 15 章,“发布或主题规划中的测试人员活动”,讨论了要保留的有用指标。
Chapter 15, “Tester Activities in Release or Theme Planning,” talks about useful metrics to keep.
您需要一些方法来了解您的团队在迭代中的任何时间点完成了多少工作,以及还有多少工作要做。您需要知道何时某些故事无法完成并且团队需要 B 计划。迭代燃尽图和任务的估计时间与实际时间是用于衡量团队进度的示例。他们可能会也可能不会为您的特定团队提供价值。
You need some way to know how much work your team has completed at any point in the iteration and an idea of how much work is left to do. You need to know when it becomes obvious that some stories can’t be completed and the team needs a Plan B. Iteration burndown charts and estimated versus actual time for tasks are examples used to measure team progress. They may or may not provide value for your particular team.
故事板或任务板是了解迭代状态的良好视觉方式,尤其是在使用颜色编码的情况下。如果太多的测试任务卡仍然在“待办事项”栏中,或者没有足够的编码任务卡被移动到“完成”或“已测试”,那么团队是时候想办法确保所有测试都完成了完全的。也许一些团队成员需要停止编码并开始承担测试任务,或者可能需要将一个故事或故事中不太重要的部分推迟到下一次迭代,以便完成对所有其他故事的测试。
Story or task boards are a good visual way to know the iteration’s status, especially if color coding is used. If too many test task cards are still in the “to do” column or not enough coding task cards have been moved to “Done” or “Tested,” it’s time for the team to think of ways to make sure all of the testing is completed. Maybe some team members need to stop coding and start taking on testing tasks, or maybe one story or a less critical part of a story needs to be put off until the next iteration so that testing for all the other stories can be finished.
这可以通过虚拟故事板和物理故事板来实现。发挥视觉效果的创意,使问题立即可见。请记住,在所有适当的级别上测试之前,任何故事都不会“完成”。团队可能对故事何时“完成”有其他标准,例如它是否经过同行评审或自动回归测试是否完成。在图 18-6所示的故事板上,每个故事行的“完成”列是最右边的一列。它左边的列是“验证”列。在所有卡片(包括测试任务卡片)都位于“完成”列中之前,故事不会被视为“完成”。看一眼棋盘就足以知道哪些故事已经完成。
This can be accomplished with virtual story boards as well as physical ones. Get creative with your visual effects so that problems are instantly visible. Remember that no story is “done” until it’s tested at all appropriate levels. Teams may have other criteria for when a story is “done,” such as whether it has been peer reviewed or the automated regression tests are completed. On the story board shown in Figure 18-6, the “Done” column for each story row is the rightmost column. The column just to the left of it is the “Verify” column. The story isn’t considered “done” until all the cards, including testing task cards, are in that “Done” column. A glance at the board is enough to know which stories are finished.
即使团队不在任务级别跟踪燃尽图,也可以在故事级别进行跟踪。了解团队每次迭代可以完成多少工作(其速度)有助于制定整体发布计划,并重新确定每次迭代的优先级。如果故事的大小趋于平均,那么知道在一次迭代中完成的故事数量就足够了。尽管计划充其量只是暂时的,但了解可以实现多少个故事是有帮助的在硬发布日期或下一季度可能完成的故事之前完成。
Even teams that don’t track burndown at the task level can do so at the story level. Knowing how much work the team can do each iteration (its velocity) helps with the overall release plan, and the reprioritizing for each iteration. It simply may be enough to know the number of stories completed in an iteration if they tend to average out to the same size. Although plans are tentative at best, it’s helpful to get an idea of about how many stories can be completed by a hard release date or what stories might get done in the upcoming quarter.
我们在第 15 章“发布或主题规划中的测试人员活动”中讨论了缺陷指标,为您提供了一些关于跟踪内容的高级概念。收集有关缺陷的指标可能非常耗时,因此在开始衡量之前始终考虑目标。您希望收集的指标的目的是什么?你需要跟风多久才能知道它们是否有用?
We talked about defect metrics in Chapter 15, “Tester Activities in Release or Theme Planning” giving you some high level ideas about what to track. Gathering metrics on defects can be very time consuming so always consider the goal before you start to measure. What is the purpose of the metrics you would like to gather? How long will you need to follow the trend before you know if they are useful?
缺陷控制始终是最喜欢捕获的指标。缺陷是什么时候发现的?在传统项目中,这要容易得多,因为您有“硬”要求和编码阶段。当整个团队都对质量负责,并且每个人都在自始至终一起工作时,就很难确定“何时”将缺陷注入系统。
Defect containment is always a favorite metric to capture. When was the defect found? In traditional projects, it is much easier as you have “hard” requirements and coding phases. When the whole team is responsible for quality, and everyone is working together throughout, it is much harder to determine “when” the defect was injected into the system.
我们想挑战这种类型的指标的想法,因为它在敏捷开发中不是必需的。但是,如果您发现有很多错误被忽略,您可能需要开始跟踪它们是什么类型的错误,以便解决根本原因。例如,如果错误可以通过单元测试发现,那么程序员可能需要更多关于编写单元测试的培训。如果错误被遗漏或误解了需求,那么可能没有足够的时间花在迭代计划上,或者验收测试不够详细。
We would like to challenge the idea of this type of metric as not necessary in agile development. However, if you find a lot of bugs are slipping through, you may want to start tracking what type of bugs they are so you can address the root cause. For example, if the bugs could have been caught with unit tests, then maybe the programmers need more training on writing unit tests. If the bugs are missed or misunderstood requirements, then maybe not enough time is spent in iteration planning, or acceptance tests aren’t detailed enough.
如果您对缺陷实行零容忍,那么您可能不需要在编码和测试期间跟踪缺陷。故事板上的一张简单卡片将为您提供所需的所有信息。
If you are practicing zero tolerance for defects, then you probably have no need to be tracking defects during coding and testing. A simple card on the story board will give you all the information you need.
无论您选择衡量什么指标,都要追求简单。
Whatever metrics you choose to measure, go for simplicity.
当指标不再有用时,不要害怕停止使用指标。如果他们最初收集的问题不再存在,就没有理由继续收集它们。
Don’t be afraid to stop using metrics when they are no longer useful. If the problem they were initially gathered for no longer exists, there is no reason to keep gathering them.
您的团队可能必须向高层管理人员或项目管理办公室 (PMO) 提供指标,尤其是当您为大型组织工作时。Patrick Fleisch,埃森哲顾问,曾在一家公司担任职能分析师在我们写这本书的时候,这家软件公司给了我们以下他的团队向他们的 PMO 提供的指标示例。
Your team may have to provide metrics to upper managers or a Project Management Office (PMO), especially if you work for a large organization. Patrick Fleisch, an Accenture Consultant who was working as a functional analyst at a software company during the time we wrote this book, gave us the following examples of metrics his team provides to their PMO.
按故事和功能区域测试执行数量
Test execution numbers by story and functional area
测试自动化状态(自动化测试与手动测试的数量)
Test automation status (number of tests automated vs. manual)
随着时间的推移通过/失败的测试数量的折线图
Line graph of the number of tests passing/failing over time
每个故事的摘要和状态
Summary and status of each story
缺陷指标
Defect metrics
收集和报告诸如此类的指标可能会导致大量开销。寻找最简单的方法来满足您的组织的需求。
Gathering and reporting metrics such as these may result in significant overhead. Look for the simplest ways to satisfy the needs of your organization.
在我们的示例迭代的这一点上,我们的敏捷测试人员与程序员、客户和其他团队成员密切合作,以小的测试-编码-审查-测试增量生成故事。要记住的几点是:
At this point in our example iteration, our agile tester works closely with programmers, customers, and other team members to produce stories in small testing-coding-reviewing-testing increments. Some points to keep in mind are:
编码和测试是迭代过程中的一个过程的一部分。
Coding and testing are part of one process during the iteration.
编码一开始就为故事编写详细的测试。
Write detailed tests for a story as soon as coding begins.
从简单的测试开始推动开发;当简单测试通过后,编写更复杂的测试用例以进一步指导编码。
Drive development by starting with a simple test; when the simple tests pass, write more complex test cases to further guide coding.
使用简单的风险评估技术来帮助集中测试工作。
Use simple risk assessment techniques to help focus testing efforts.
当要求不明确或意见不同时,使用“三的幂”。
Use the “Power of Three” when requirements aren’t clear or opinions vary.
专注于一次完成一个故事。
Focus on completing one story at a time.
与程序员密切合作,以便集成测试和编码。
Collaborate closely with programmers so that testing and coding are integrated.
批评产品的测试是开发的一部分。
Tests that critique the product are part of development.
在整个迭代过程中让客户了解情况;让他们早点经常复习。
Keep customers in the loop throughout the iteration; let them review early and often.
团队中的每个人都可以从事测试任务。
Everyone on the team can work on testing tasks.
Testers can facilitate communication between the customer team and development team.
确定对你的团队来说最好的“错误修复”选择是什么,但一个好的目标是在发布时没有错误。
Determine what the best “bug fixing” choice for your team is, but a good goal is to aim to have no bugs by release time.
将新的自动化测试添加到回归套件中,并安排它经常运行以提供足够的反馈。
Add new automated tests to the regression suite and schedule it to run often enough to provide adequate feedback.
在对所有应用程序进行编码后,手动探索性测试有助于找到缺失的需求。
Manual exploratory testing helps find missing requirements after all the application has been coded.
与其他专家合作以获得完成测试所需的资源和基础设施。
Collaborate with other experts to get the resources and infrastructure needed to complete testing.
考虑迭代期间需要哪些指标;进度和缺陷指标是两个例子。
Consider what metrics you need during the iteration; progress and defect metrics are two examples.
我们已经完成了一次迭代。当团队结束本次迭代并为下一次做准备时,测试人员会做什么?我们喜欢专注于我们和我们团队的其他成员下次如何改进和交付更好的产品。
We’ve completed an iteration. What do testers do as the team wraps up this iteration and prepares for the next? We like to focus on how we and the rest of our team can improve and deliver a better product next time.
敏捷开发的乐趣之一是有机会在每次迭代结束时向客户展示已完成的故事。客户可以看到真实、实时、工作的应用程序。他们可以提出问题并提供反馈。参与该项目的每个人,无论是业务方面还是技术方面,都会获得成就感。
One of the pleasures of agile development is the chance to show completed stories to customers at the end of each iteration. Customers get to see a real, live, working application. They get to ask questions and give feedback. Everyone involved in the project, from both the business and technical sides, gets to enjoy a sense of accomplishment.
在 Lisa 的团队中,测试人员进行迭代评审。在所有团队成员中,他们通常处理的故事最多。他们天生就是信息提供者,他们很清楚客户需要了解新功能的哪些方面。让测试人员展示可交付成果是一种常见的做法,尽管没有硬性规定。团队中的业务专家也是进行演示的不错选择,因为他们最了解软件如何满足业务需求,并且他们会对产品有更大的主人翁感。ScrumMaster、程序员或业务分析师可以演示新功能并且经常这样做。珍妮特鼓励轮换这一荣誉。
On Lisa’s team, the testers conduct the iteration review. Among all the team members, they’ve usually worked on the most stories. They have a natural role as information providers, and they have a good idea what the customers need to know about the new functionality. Having testers show off the deliverables is a common practice, although there is no hard and fast rule. The business experts on the team are a good choice for conducting the demo too, because they have the best understanding of how the software meets the business needs and they’ll feel greater ownership of the product. The ScrumMaster, a programmer, or a business analyst could demonstrate the new features and often does. Janet encourages rotating this honor.
任何人都可能注意到客户在参与演示时发表的评论,但测试人员是不错的人选。随着演示的进行,他们可能会注意到以前未检测到的不一致。随着问题的出现,客户可能会决定他们想要改变一些小的东西,比如帮助文本,或者更大的东西,比如一个特性的行为方式。微小的变化通常可以变成任务并在下一次迭代中处理,但有些变化大到足以变成故事来计划未来的发布。
Anyone may note the comments made by customers as they participate in the demo, but testers are good candidates. They may notice previously undetected inconsistencies as the demo progresses. As questions come up, customers might decide they want to change something minor, such as help text, or something bigger, such as how a feature behaves. Minor changes can usually be made into tasks and dealt with in the next iteration, but some changes are big enough to turn into stories to plan into future releases.
迭代演示(在 Scrum 世界中称为冲刺评审)是让每个人讨论和思考应用程序的绝佳机会。利用它。审查会议通常很短,可能不到半小时。如果在展示新故事后还有剩余时间,请询问客户是否遇到过他们未报告的先前版本的任何问题。他们是否有任何一般性顾虑,是否需要帮助以了解如何使用某项功能,或者是否出现了任何新问题?当然,您可以随时与客户交谈,但是让大多数利益相关者与开发团队同在一个房间可以产生有趣的想法。
Iteration demos (called sprint reviews in the Scrum world) are a super opportunity to get everyone talking and thinking about the application. Take advantage of it. Review meetings are usually short and can be under half an hour. If there’s time left over after demonstrating new stories, ask customers if they’ve experienced any problems with the previous release that they haven’t reported. Do they have any general concerns, do they need help understanding how to use a feature, or have any new issues arisen? Of course, you can talk to customers anytime, but having most of the stakeholders in the room with the development team can lead to interesting ideas.
敏捷开发意味着不断改进您的工作方式,而回顾是开始确定您可以做得更好的地方以及如何做得更好的好地方。我们建议在每次迭代和发布周期结束时花点时间回顾并讨论哪些进展顺利,哪些进展不顺利,以及您可能希望在下一次迭代中尝试哪些内容。进行回顾会议有不同的方法。无论您使用哪种方法,关键是每个团队成员都感到安全,每个人都受到尊重,并且没有指责或责备。
Agile development means continually improving the way you work, and retrospectives are an excellent place to start identifying what and how you can do better. We recommend taking time at the end of each iteration and release cycle to look back and talk about what went well, what didn’t, and what you might like to try in the next iteration. There are different approaches for conducting retrospective sessions. No matter what approach you use, it’s key that each team member feels safe, everyone is respected, and there’s no finger-pointing or blame.
The whole idea is to make the process better, one baby step at a time.
迭代回顾中使用的一种常见练习是“开始、停止、继续”。团队自问:“在过去的迭代中,哪些进展顺利?发生了不该再发生的事?我们可以开始做什么来帮助处理不顺利的事情?” 每个团队成员都可以建议开始做些什么来改进,停止做没有效果的事情,以及应该继续做的有帮助的事情。协调员或 ScrumMaster 将它们列在白板或大纸上。将它们张贴在每个人都可以在迭代期间再次阅读的位置。图 19-1显示了正在进行的“停止、开始和继续”回顾。ScrumMaster(站立)正在故事板上的大纸上写下停止、开始和继续建议。
One common exercise used in iteration retrospectives is “start, stop, continue.” The team asks itself: “What went well during this past iteration? What happened that shouldn’t happen again? What can we start doing to help with things that didn’t go well?” Each team member can suggest things to start doing to improve, things to stop doing that weren’t working, and things that are helping that should be continued. A facilitator or ScrumMaster lists them on a whiteboard or big piece of paper. Post them in a location where everyone can read them again during the iteration. Figure 19-1 shows a “stop, start, and continue” retrospective in progress. The ScrumMaster (standing) is writing stop, start, and continue suggestions on the big piece of paper on the story board.
Agile Retrospectives: Making Good Teams Great [2006] 具有使回顾更有成效的富有想象力的想法(参见参考书目)。
Agile Retrospectives: Making Good Teams Great [2006] has imaginative ideas for making retrospectives more productive (see the bibliography).
一些团队提前开始这个过程。所有团队成员在便利贴上写下“开始”、“停止”和“继续”项目,然后在回顾会议期间,他们将便利贴贴在黑板上并按主题分组。“开始、停止、继续”只是您可能使用的术语之一。其他一些想法是:“进展顺利的事情”、“需要改进的事情”、“愉快的事情”、“令人沮丧的事情”和“尝试一下”。使用适合您的任何名称。有可能很难记住过去两周,更不用说整个版本了,如果那是你的回顾涵盖的内容。研究不同的创造性方法来反映您团队的经验。
Some teams start this process ahead of time. All team members write “start,” “stop,” and “continue” items on sticky notes, and then during the retrospective meeting they put the stickies on the board and group them by topic. “Start, stop, continue” is just one example of the terms you might use. Some other ideas are: “Things that went well,” “Things to improve,” “Enjoyable,” “Frustrating,” and “To Try.” Use whatever names that work for you. It can be hard to remember the past two weeks, much less an entire release, if that’s what your retrospective covers. Research different creative approaches to reflecting on your team’s experiences.
这是 Lisa 团队的“停止、开始、继续”列表示例:
Here’s a sample “stop, start, continue” list from Lisa’s team:
开始:
Start:
提前向我们发送下一个 sprint 的故事。
Sending out next sprint’s stories to us earlier.
不要进行懒惰的单记录处理。将每个服务调用都视为远程调用。
Don’t do lazy, single-record processing. Think of every service call as a remote call.
将任何数据库更改传达给每个人。
Communicate any database changes to everyone.
停止:
Stop:
接受没有完整要求的故事。
Accepting stories without complete requirements.
继续:
Continue:
为您正在处理的代码运行 FitNesse 测试。
Running FitNesse tests for the code you’re working on.
记录会议或非正式讨论中出现的内容。
Documenting what came up in meeting or informal discussions.
彼此更好地沟通。
Communicating better with each other.
尽早展示模型。
Showing mock-ups early.
进行 FitNesse 驱动的开发。
Doing FitNesse driven development.
如果“开始、停止、继续”项目列表很长,最好选择一两个项目作为新迭代的重点。为了确定项目的优先级,给每个团队成员“n”票,他们可以分配给项目。Lisa 团队中的十个人每人获得三票,如果他们认为最重要,他们可以将所有票投给一个项目,或者他们可以投票给两个或三个不同的项目。得票最多的项目被标记为焦点项目。珍妮特也通过这种确定优先次序的方式取得了成功。
If the list of “start, stop, continue” items is long, it’s a good idea to choose one or two to focus on for the new iteration. To prioritize the items, give each team member “n” votes they can assign to items. The ten people on Lisa’s team each get three votes, and they can apply them all to one item if they feel that’s most important, or they can vote for two or three different items. The items with the most votes are noted as the focus items. Janet has had success with this way of prioritizing as well.
除了“开始、停止、继续”项目之外,团队还可以简单地为下一次迭代要采取的行动编写任务卡。例如,如果正在进行的构建速度太慢,请写一张卡片“在十分钟内完成正在进行的构建”。
In addition to “start, stop, continue” items, the team may simply write task cards for actions to be undertaken the next iteration. For example, if the ongoing build is too slow, write a card to “get ongoing build under ten minutes.”
在下一次迭代中,花一些时间查看您想要改进的一两个重点项目。在那次迭代结束时,做一个检查点看看你是否有所改进。如果不是,请问为什么。你应该尝试不同的东西吗?它还重要吗?可能是它的重要性下降了,或者从大局来看真的不重要。如果你认为你在某个问题领域有所改进,但它又重新浮出水面,你将不得不决定对此做些什么,否则就不再谈论它。
In the next iteration, take some time to look at the one or two focus items you wanted to improve. At the end of that iteration, take a checkpoint to see if you improved. If not, ask why. Should you try something different? Is it still important? It could be it has dropped in importance or really wasn’t important in the big picture. If you thought you improved on a problem area and it resurfaces, you’ll have to decide to do something about it or else quit talking about it.
我们发现回顾是团队识别和解决问题的一种简单而高效的方法。回顾会议是提出测试相关问题的绝好机会。以客观、非责备的方式提出问题。团队可以讨论每个问题,可能导致问题的原因,并写下一些解决问题的想法。
We’ve found that retrospectives are a simple and highly effective way for teams to identify and address issues. The retrospective meeting is a perfect opportunity to raise testing-related issues. Bring up the issues in an objective, non-blaming way. The team can discuss each problem, what might be causing it, and write down some ideas to fix it.
让我们来看看其中一些被列入改进清单的项目。很多时候,一个团队会发现非常大的问题,但从不跟进并实际采取措施解决它们。例如,在程序员声称编码完成后,可能会发现很多单元级错误。
Let’s take a look at some of those items that made it onto the list for improvement. Too many times, a team will identify really big issues but never follow up and actually do something about them. For example, maybe a lot of unit-level bugs are discovered after the programmers have claimed coding was complete.
团队可能会认为程序员没有用单元测试覆盖足够的代码。他们可能会在签入新代码之前编写一个操作项来运行代码覆盖工具,或者开始为每个故事编写“单元测试”任务卡以确保它们已完成。也许团队在迭代结束之前没有完成所有的测试自动化任务。当他们讨论这个问题时,团队发现最初的可执行测试过于复杂,他们需要先专注于编写和自动化一个简单的测试,或者配对以获得更好的测试设计。确保行动项目是具体的。
The team may decide the programmers aren’t covering enough code with unit tests. They might write an action item to run the code coverage tool before they check in new code, or start writing a “unit tests” task card for each story to make sure they’re completed. Perhaps the team didn’t finish all the test automation tasks before the iteration ended. As they discuss the problem, the team finds that the initial executable tests were too complex, and they need to focus on writing and automating a simple test first, or pair for a better test design. Make sure the action items are concrete.
敏捷团队尝试解决他们自己的问题并制定指导方针来帮助他们自己改进。针对一个问题的行动项目可能对其他问题有帮助。当 Lisa 的团队在完成故事并在每次迭代中对其进行测试时遇到困难时,他们在几次回顾的过程中提出了各种规则:
Agile teams try to solve their own problems and set guidelines to help themselves improve. Action items aimed at one problem may help with others. When Lisa’s team had trouble finishing stories and getting them tested during each iteration, it came up with various rules over the course of a few retrospectives:
在迭代的第四天之前完成所有故事的高级测试用例。
Finish high-level test cases for all stories by the fourth day of the iteration.
在迭代的第四天交付一个故事进行测试。
Deliver one story to test by the fourth day of the iteration.
专注于一次完成一个故事。
Focus on finishing one story at a time.
100% 的功能必须在迭代的倒数第二天下班前签入。
100% of features must be checked in by close of business on the next-to-last day of the iteration.
这些规则不仅仅帮助团队完成了测试任务。他们促进了流程和节奏,帮助团队在每次迭代过程中以稳定、可持续的速度工作。
These rules did more than help the team finish testing tasks. They facilitated a flow and rhythm that helped the team work at a steady, sustainable pace over the course of each iteration.
通过回顾行动项目来开始下一次回顾会议,看看哪些项目是有益的。丽莎的团队将快乐、悲伤或中性的面孔放在旁边项目来表示团队是否尝试过它们并发现它们是成功的。团队应该找出任何悲伤面孔背后的原因。是不是有些项目被遗忘了?时间限制是否阻止了团队尝试新活动?只是后来似乎不是一个好主意吗?这些讨论可能会导致更改改进项目或将其演化为新项目。
Begin the next retrospective meeting by reviewing the action items to see what items were beneficial. Lisa’s team puts happy, sad, or neutral faces next to items to denote whether the team tried them and found them successful. The team should figure out the reasons behind any sad faces. Were some items simply forgotten? Did time constraints keep the team from trying a new activity? Did it just seem to be less of a good idea later? These discussions might lead to changing the improvement item or evolving it into a new one.
当改进行动成为团队的习惯时,它们就不再需要写在“停止、开始和继续”列表上。运行良好的“开始”项目可能会移至“继续”列。有些想法行不通,或者被证明是不必要的,这些也可以从下一次迭代的列表中删除。
When the actions for improvement become a habit to the team, they no longer need to be written on the “stop, start, and continue” list. “Start” items that work well may be moved to the “Continue” column. Some ideas don’t work, or prove to be unnecessary, and those can also be taken off the list for the next iteration.
在迭代期间参考您的改进想法和行动项目。将它们张贴在每个人都经常看到的位置(墙上或网上)。Lisa 的团队有时会在迭代中期的站立会议中仔细检查列表。如果您在迭代过程中想到新的改进想法,请将它们写下来,甚至可能在现有列表中,这样您就不会在下一次迭代时忘记。
Refer to your ideas for improvement and action items during the iteration. Post them in a location (on a wall or online) where everyone sees them often. Lisa’s team sometimes goes through the list during a mid-iteration stand-up meeting. If you think of new improvement ideas during the iteration, write them down, possibly even on the existing list, so you won’t forget for the next iteration.
在整个迭代过程中跟踪阻碍您前进的事情是个好主意。在一些大的可见图表上保留障碍积压。讨论每次迭代中的障碍,写任务卡或采取行动消除它们。
It’s a good idea to keep track of things that get in your way throughout the iteration. Keep an impediment backlog on some big visible chart. Talk about the impediments in each iteration, and write task cards or take action to eliminate them.
使用回顾作为提出与测试相关的问题并让整个团队思考可能的解决方案的机会。我们对整个团队提出的创新想法感到惊喜,他们专注于如何改进其工作方式。
Use retrospectives as an opportunity to raise testing-related issues and get the whole team thinking about possible solutions. We’ve been pleasantly surprised with the innovative ideas that come out of an entire team focusing on how to improve the way it works.
敏捷开发实践倾向于缓和更传统或混乱过程中存在的高潮和低谷。如果您的瀑布团队在长达一年的周期以两个月的紧张修复和测试周期结束后最终设法推出了一个版本,那么每个人都可能准备好举办一个盛大的派对来庆祝这一事件——或者他们可能只是崩溃了几个星期。每两周发布一次的敏捷团队倾向于保持正常的编码和测试状态,在吸取足够的呼吸以进行迭代审查和回顾后开始下一组故事。这很好,但你知道他们怎么说只工作不玩耍。
Agile development practices tend to moderate the highs and lows that exist in more traditional or chaotic processes. If your waterfall team finally manages to push a release out the door after a year-long cycle ending in a two-month stressful fix-and-test cycle, everyone may be ready to celebrate the event with a big party—or they might just collapse for a couple of weeks. Agile teams that release every two weeks tend to stay in their normal coding and testing groove, starting on the next set of stories after drawing just enough breath to hold an iteration review and retrospective. This is nice, but you know what they say about all work and no play.
确保你的团队至少花一点时间来表扬自己并认可他们的成就。即使是很小的成功也应该得到奖励。享受是一种至关重要的敏捷价值,一点点动力可以帮助您的团队在成功的道路上继续前进。出于某种原因,这可能很难做到。许多敏捷团队很难花时间庆祝成功。有时您急于开始下一次迭代,而没有花时间为之前的成就祝贺自己。
Make sure your team takes at least a little time to pat itself on the back and recognize its achievements. Even small successes deserve a reward. Enjoyment is a vital agile value, and a little motivation helps your team continue on its successful path. For some reason, this can be hard to do. Many agile teams have trouble taking time to celebrate success. Sometimes you’re eager to get going with the next iteration and don’t take time to congratulate yourselves on the previous accomplishments.
Lisa 的团队每隔一个星期四结束一次迭代,并在第二天进行回顾、迭代审查和发布。会议结束后,他们通常会进行他们称之为“周五娱乐”的活动。这有时包括玩一些愚蠢的琐事或棋盘游戏、出去喝一杯或打一场迷你高尔夫球。有机会放松和大笑有利于团队建设。
Lisa’s team ends an iteration every other Thursday and conducts its retrospective, iteration review, and release the following day. After their meetings conclude, they usually engage in something they call “Friday Fun.” This sometimes consists of playing a silly trivia or board game, going out for a drink, or playing a round of miniature golf. Getting a chance to relax and have a good laugh has a team-building side benefit.
对于更大的里程碑,例如重大发布或实现测试覆盖率目标,整个公司都会举行派对庆祝,带来餐饮或外出星期五下午去一家餐馆。这是对业务和技术团队中每个人的一个很好的奖励和认可。
For bigger milestones, such as a big release or achieving a test coverage goal, the whole company has a party to celebrate, bringing in catered food or going out to a restaurant on Friday afternoon. This is a nice reward and recognizes for everyone on both the business and technical teams.
如果您是一个新的敏捷团队,请通过奖励小成就来激励自己。为每个构建中通过的单元测试数量增加而欢呼。图表显示实际燃烧量与预计燃烧量相匹配。当构建中损坏的单元测试被修复时按铃(好吧,那个可能很烦人,但以某种方式识别它。)
If yours is a new agile team, motivate yourselves by rewarding small accomplishments. Cheer the rising number of unit tests passing in each build. Oooh and aaah over the chart showing actual burn down matching the projected burn down. Ring a bell when the broken unit tests in the build are fixed (okay, that one might be annoying, but recognize it in some way.)
也要庆祝你个人的成功。祝贺您的同事完成了项目的第一个性能测试基线。给你的 DBA 一个金星来实现生产备份系统。奖励自己解决那个困难的测试自动化问题。下次与客户会面时带上饼干。认识为您提供 JavaScript 工具以加速某些 GUI 验证测试的程序员。动用你的想象力。
Celebrate your individual successes, too. Congratulate your coworker for completing the project’s first performance test baseline. Give your DBA a gold star for implementing a production back-up system. Give yourself a treat for solving that hard test-automation problem. Bring cookies to your next meeting with the customers. Recognize the programmer who gave you a JavaScript harness that sped up testing of some GUI validations. Use your imagination.
花时间庆祝成功可以让您的团队后退一步,获得全新的视角,并重新焕发活力,从而不断改进您的产品。让团队成员有机会欣赏彼此的贡献。不要陷入每个人都一直低着头工作的例行公事。
Taking time to celebrate successes lets your team take a step back, get a fresh perspective, and renew its energy so it can keep improving your product. Give team members a chance to appreciate each other’s contributions. Don’t fall into a routine where everyone has their head down working all the time.
在敏捷开发中,我们有机会在每次短迭代结束时停下来并获得新的视角。我们可以进行细微的课程修正,决定尝试一种新的测试工具,想出更好的方法来从客户那里引出示例,或者确定对特定类型测试专业知识的需求。
In agile development, we get a chance to stop and get a new perspective at the end of each short iteration. We can make minor course corrections, decide to try out a new test tool, think of better ways to elicit examples from customers, or identify the need for a particular type of testing expertise.
在本章中,我们研究了一些结束迭代或发布的活动。
In this chapter, we looked at some activities to wrap up the iteration or release.
迭代评审是从客户团队获得反馈和输入的绝佳机会。
The iteration review is an excellent opportunity to get feedback and input from the customer team.
回顾是帮助您的团队改进的重要实践。
Retrospectives are a critical practice to help your team improve.
查看团队可以改进的所有领域,但一次只关注一两个领域。
Look at all areas where the team can improve, but focus on one or two at a time.
找到一种在迭代过程中牢记改进项的方法。
Find a way to keep improvement items in mind during the iteration.
庆祝大大小小的成功,并认可不同角色和活动的贡献。
Celebrate both big and small successes, and recognize the contributions from different roles and activities.
利用每次迭代后的机会来识别与测试相关的障碍,并想办法克服它们。
Take advantage of the opportunity after each iteration to identify testing-related obstacles, and think of ways to overcome them.
在本章中,我们将分享您作为测试人员可以做些什么来帮助您的团队和您的组织成功交付高质量的产品。相同的流程和工具可用于收缩包装产品、定制解决方案或内部开发产品。敏捷测试人员可以做出独特的贡献,帮助客户和开发团队定义和产生业务所需的价值。
In this chapter, we share what you as a tester can do to help your team and your organization successfully deliver a high-quality product. The same process and tools can be used for shrink-wrapped products, customized solutions, or internal development products. Agile testers can make unique contributions that help both the customer and developer team define and produce the value that the business needs.
许多关于敏捷开发的书籍都在谈论实际的开发周期,但忽略了如何制造产品以及成功交付该产品需要什么。仅仅编写代码、测试并说出它是不够的完毕。这就像从商店买东西:如果购买时有很好的服务,您再去那里购买的可能性有多大?
Many of the books on agile development talk about the actual development cycle but neglect to talk about what makes a product and what it takes to successfully deliver that product. It’s not enough to just code, test, and say it’s done. It’s like buying something from a store: If there is great service to go with the purchase, how much more likely are you to go back and buy there again?
我们的目标是及时为企业创造价值。我们不仅要满足要求,还要取悦我们的客户。在我们发布之前,我们要确保所有可交付成果都已准备就绪并得到适当完善。希望您尽早开始计划,不仅要满足代码要求,还要计划培训、文档以及制作高价值产品所需的一切。
Our goal is to deliver value to the business in a timely manner. We don’t want just to meet requirements but also to delight our customers. Before we release, we want to make sure all of the deliverables are ready and polished up appropriately. Hopefully, you started planning early to meet not only the code requirements but to plan for training, documentation, and everything that goes into making a high-value product.
想象一下您正在为您的发布做好生产准备。您刚刚完成了最后一次迭代并结束了最后一个故事测试。您的自动回归套件一直在每个新构建上运行,或者至少在每个夜间构建上运行。你现在做什么将取决于你的过程有多严格。如果您一直对错误保持“零容忍”,那么您的状态可能非常好。
Imagine yourself in the middle of getting your release ready for production. You’ve just finished your last iteration and are wrapping up your last story test. Your automated regression suite has been running on every new build, or at least on every nightly build. What you do now will depend on how disciplined your process has been. If you’ve been keeping to the “zero tolerance” for bugs, you’re probably in pretty good shape.
如果您是那些认为可以将错误留到最后修复的团队之一,那么您的状态可能不太好,可能需要引入迭代以“强化”或修复错误。我们不建议这样做,但如果您的团队在开发周期中引入了很多突出的错误,则您需要在进入最终游戏之前解决这些错误。我们发现新团队往往会陷入这个陷阱。
If you’re one of those teams that thinks you can leave bugs until the end to fix, you’re probably not in such good shape and may need to introduce an iteration for “hardening” or bug fixes. We don’t recommend this, but if your team has a lot of outstanding bugs that have been introduced during the development cycle, you need to get those addressed before you go into the end game. We find that new teams tend to fall into this trap.
此外,任何版本都有许多不同的组件,一些在软件中,一些不在软件中。您的客户需要安装和学习使用新功能。考虑所有对成功发布至关重要的元素,因为是时候结束所有这些未解决的问题并磨练您的产品了。
In addition, there are lots of varied components to any release, some in the software, some not. You have customers who need to install and learn to use the new features. Think about all those elements that are critical to a successful release, because it’s time to wrap up all those loose ends and hone your product.
Bob Galen 是一名敏捷教练和终局专家,他观察到敏捷开发可能并没有渗透到组织的每个角落。他指出,“在软件的物理交付方面,敏捷测试人员可以充当管道或促进者。”
Bob Galen, an agile coach and end-game expert, observes that agile development may not have seeped into every organizational nook and cranny. He notes, “Agile testers can serve as a conduit or facilitator when it comes to physical delivery of the software.”
因为测试和编码是敏捷开发中一个过程的一部分,所以我们不希望为额外的测试时间制定特殊的计划,但在现实生活中我们可能需要一些额外的时间。
Because testing and coding are part of one process in agile development, we’d prefer not to make special plans for extra testing time, but in real life we might need some extra time.
大多数团队都会积累一些技术债务,尽管本意是好的,尤其是当他们使用遗留代码时。为了保持速度,您的团队可能需要定期计划重构迭代以添加测试、升级工具和减少技术债务。Lisa 的团队大约每六个月进行一次重构冲刺。虽然业务通常不会在重构冲刺结束时获得任何直接收益,但业务专家明白这些特殊冲刺会带来更好的测试覆盖率、为未来开发打下坚实的基础、减少技术债务以及更高的整体团队速度.
Most teams accumulate some technical debt, despite the best intentions, especially if they’re working with legacy code. To maintain velocity, your team may need to plan a refactoring iteration at regular intervals to add tests, upgrade tools, and reduce technical debt. Lisa’s team conducts a refactoring sprint about every six months. While the business doesn’t usually receive any direct benefits at the end of a refactoring sprint, the business experts understand that these special sprints result in better test coverage, a solid base for future development, reduced technical debt, and a higher overall team velocity.
一些团队求助于“强化”迭代,他们只花时间查找和修复错误,而不引入任何新功能。这是保持应用程序及其基础架构稳固的最后手段。新团队可能需要额外的迭代来完成测试任务,如果是这样,他们会在发布计划中为此预算时间。
Some teams resort to “hardening” iterations, where they spend time only finding and fixing bugs, and they don’t introduce any new functionality. This is a last resort for keeping the application and its infrastructure solid. New teams may need an extra iteration to complete testing tasks, and if so, they budget time for that in the release plan.
使用回顾和其他流程改进实践来学习集成测试和编码的方法,以便每次迭代中生成的代码都可以投入生产。实现该目标后,努力确保每天都有可以发布到生产环境的稳定构建。Lisa 的团队成员认为,在他们努力在发布前获得任何稳定版本的日子里,这是一个无法实现的目标,但仅仅过了几年,几乎每个版本都值得发布。
Use retrospectives and other process improvement practices to learn ways to integrate testing and coding so that the code produced in each iteration is production-ready. When that goal is achieved, work to ensure that a stable build that could be released to production is available every day. Lisa’s team members thought that this was an unattainable goal in the days when they struggled to get any stable build before release, but it was only a couple of years before almost every build was release-worthy.
当您的构建稳定后,您就可以进入“终结游戏”了。
When your build is stable, you are ready to enter the “End Game.”
什么是终极游戏?我们听说人们在交付很多东西之前就把时间称为交付时间,但“结束游戏”似乎最合适。这是团队对产品进行最后润色的时候。你正在点你的 i 并穿过你的 t。这是交付终点线前的最后一段路段。这并不是一个错误修复周期,因为到那时你不应该有任何未解决的错误,但这并不意味着你可能没有一两个要修复的。
What is the end game? We’ve heard people call the time right before delivery many things, but the “end game” seems to fit best. It’s the time when the team applies the finishing touches to the product. You’re dotting your i’s and crossing your t’s. It’s the last stretch before the delivery finish line. It’s not meant to be a bug-fix cycle, because you shouldn’t have any outstanding bugs by then, but that doesn’t mean you might not have one or two to fix.
您的组织中可能有您没有参与早期计划的小组。现在是时候与管理暂存和生产环境的人员、配置经理、团队外的数据库管理员以及在将软件从开发转移到暂存和生产过程中发挥作用的每个人密切合作了。如果这次您没有尽早与他们合作,请考虑在下一次发布计划会议期间与他们交谈,并在整个开发周期中与他们保持联系。
You might have groups in your organization that you didn’t involve in your earlier planning. Now it’s time to work closely with the folks that administer the staging and production environments, the configuration managers, the database administrators outside of your team, and everyone who plays a role in moving the software from development to staging and production. If you weren’t working with them early this time, consider talking to these folks during your next release planning sessions, and keep in touch with them throughout the development cycle.
Bob Galen 告诉我们,他团队中的测试人员与管理暂存和生产环境的运营团队合作。由于运营团队位于远程,因此发现敏捷团队的指导特别有价值。
Bob Galen tells us that the testers on his team have partnered with the operations group that manages the staging and production environments. Because the operations group is remote, it finds that having guidance from the agile team is particularly valuable.
总有一些系统级测试无法自动化,或者不值得自动化。通常情况下,您的暂存环境是唯一的地方您可以在其中进行一些系统级集成测试或系统级负载和压力测试。我们建议您在开发完成后分配一些时间来完成这些类型的完成任务。不要一直写到最后。
There are always system-level tests that can’t be automated, or are not worth automating. More often than not, your staging environment is the only place where you can do some system-level integration tests or system-level load and stress testing. We suggest that you allot some time after development for these types of finishing tasks. Don’t code right up to the end.
根据需要为结束游戏计划尽可能多的时间。Janet 发现,结束游戏所需的时间长短随团队的成熟度和应用程序的大小而变化。可能只需要一天就可以完成额外的任务,但也可能需要一周,有时甚至需要整整两周的迭代。第 12 章“测试象限总结”中使用的示例中的团队安排了两周时间,因为这是一个复杂的系统,需要相当多的设置和系统测试。
Plan as much time for the end game as you need. Janet has found that the length of time needed for the end game varies with the maturity of the team and the size of the application. It may be that only one day is needed to finish the extra tasks, but it may be one week or sometimes as much as a whole two-week iteration. The team from the example used in Chapter 12, “Summary of Testing Quadrants,” scheduled two weeks, because it was a complex system that required a fair bit of setup and system testing.
如果您有一个大型组织,您可能有十个或十五个团队为单个产品或同一应用程序的不同功能区域开发软件。这些领域或者产品可能都需要一起发布,所以一个整合的端游是很有必要的。这并不意味着您将集成留到最后。在您的整个开发周期中,与其他团队的协调至关重要,如果您有测试集成系统,我们建议您确保在游戏结束之前很久就尝试过集成。
If you have a large organization, you might have ten or fifteen teams developing software for individual products or for separate areas of functionality for the same application. These areas or products may all need to release together, so an integrated end game is necessary. This does not mean that you leave the integration until the very end. Coordination with the other teams will be critical all along your development cycle, and if you have a test integration system, we recommend that you be sure that you have tried to integrate long before the end game.
您也可能有团队以外的考虑,例如,在企业级别使用外部团队交付的软件。
You also may have considerations beyond your team, for example, working with software delivered by external teams at the enterprise level.
利用这个游戏结束时间进行一些最终的探索性测试。退一步看整个系统,做一些端到端的场景。此类测试将确认应用程序是否正常工作,让您对产品更有信心,并为下一次迭代或发布提供信息。
Use this end-game time to do some final exploratory testing. Step back and look at the whole system and do some end-to-end scenarios. Such testing will confirm that the application is working correctly, give you added confidence in the product, and provide information for the next iteration or release.
我们建议对每个候选发布版本进行自动回归测试。如果您按照我们的建议对每个新构建持续运行自动回归测试,或者至少每天运行一次,那么您已经这样做了。如果您的某些回归测试是手动的,则您需要为这些测试计划时间,否则它们可能无法完成。如果有多个发布候选版本,则基于对每个构建所做的更改的风险评估将确定需要运行哪些测试。
We recommend that the automated regression testing be done against every release candidate. If you’re following our recommendation to run automated regression tests continually on each new build, or at least daily, you’ve already done this. If some of your regression tests are manual, you’ll need to plan time for those or they might not get done. A risk assessment based on changes made to each build will determine what tests need to be run if there is more than one release candidate.
无论您使用的是传统开发流程还是敏捷开发流程,模拟生产的暂存环境对于发布前的最终测试以及测试发布流程本身都是至关重要的。作为最终游戏的一部分,您的应用程序应该部署到暂存区,就像您将其部署到生产环境中一样,或者就像您的客户在他们的环境中一样。在 Janet 见过的许多组织中,暂存环境通常在多个项目之间共享,部署必须作为发布计划的一部分进行安排。提前考虑如何处理依赖性、使用暂存环境与其他团队集成以及与外部第三方合作。这可能感觉像是“传统的”测试计划,但您可能正在与尚未采用敏捷开发的团队打交道。
Whether you are using traditional or agile development processes, a staging environment that mimics production is vital for final testing before release, as well as for testing the release process itself. As part of the end game, your application should be deployed to staging just like you would deploy it to production, or as your customers would on their environments. In many organizations that Janet has seen, the staging environment is usually shared among multiple projects, and the deployment must be scheduled as part of the release planning. Consider ahead of time how to handle dependencies, integrating with other teams using the staging environment, and working with external third parties. It might feel like “traditional” test planning, but you might be dealing with teams that haven’t embraced agile development.
尽管敏捷促进持续集成,但通常很难与项目控制之外的第三方产品或其他应用程序集成。暂存环境可以有更好的控制,以便外部应用程序可以连接并访问第三方测试环境。暂存环境还可用于负载和性能测试、模拟部署、故障转移测试以及手动回归测试和探索性功能测试。环境之间始终存在配置差异,因此您的暂存环境是测试这些的好地方。
Although agile promotes continuous integration, it is often difficult to integrate with third-party products or other applications outside your project’s control. Staging environments can have better controls so that external applications may connect and have access to third-party test environments. Staging environments can also be used for load and performance testing, mock deploys, fail-over testing, and manual regression tests and exploratory functional testing. There are always configuration differences between environments so your staging environment is a good place to test these.
应在整个项目中针对您正在开发的应用程序的特定部分安排负载测试。如果您的暂存环境需求量很大,您可能无法在游戏结束前进行完整的系统负载测试。
Load testing should be scheduled throughout the project on specific pieces of the application that you are developing. If your staging environment is in high demand, you may not be able to do full system load testing until the end game.
到这个时候,您应该能够对所有产品功能进行长期运行的可靠性测试。检查正常负载下的崩溃和性能下降。在发布时完成时,它应该只是最终确认。
By this time, you should be able to do long-running reliability tests on all product functionality. Check for crashes and degradation of performance with normal load. When done at release time, it should be a final confirmation only.
容错和恢复测试也最好在您的暂存环境中完成,因为测试环境通常没有必要的设置。出于同样的原因,您可能只能测试某些方面的安全性。一个例子是 https,一种通过加密安全套接字的安全 http 连接。一些组织可能会选择仅在其暂存环境中拥有必要的证书。其他示例是集群或数据复制。确保让所有需要参与此测试的各方都参与进来。
Fault tolerance and recovery testing is best done on your staging environment as well, because test environments usually don’t have the necessary setup. For these same reasons, you may only be able to test certain aspects of security. One example is https, a secure http connection through encrypted secure sockets. Some organizations may choose to have the necessary certificates on their staging environment only. Other examples are clustering or data replication. Make sure you involve all parties who need to be included in this testing.
您的团队可能敏捷,但您组织中的其他产品团队或与您的团队合作的第三方可能不是。
Your team may be agile, but other product teams in your organization, or third parties your team works with, may not be.
提前与其他产品团队或拥有需要与您的产品集成的产品的外部合作伙伴进行良好协调。如果您及早发现了这些风险并尽可能多地进行了前期测试,那么在游戏结束时进行的测试应该只是最终验证。但是,总有最后一刻的惊喜,因此您可能需要准备好对您的应用程序进行更改。
Coordinate well in advance with other product teams or outside partners that have products that need to integrate with your product. If you have identified these risks early and done as much up-front testing as possible, the testing done during the end game should be final verification only. However, there are always last-minute surprises, so you may need to be prepared to make changes to your application.
在开发过程中用于测试的模拟器和模拟对象等工具可以帮助减轻一些风险,但越早使用外部应用程序进行测试,风险就越低。
Tools like simulators and mock objects used for testing during development can help alleviate some of the risks, but the sooner you can test with external applications, the lower the risk.
在开发应用程序时,我们会更改字段、在数据库中添加列或删除过时的列。不同的团队以不同的方式解决这个问题。一些团队在每次新构建时都重新创建数据库。这适用于新应用程序,因为没有现有数据。但是,当应用程序存在于生产中并具有关联数据后,这种方法将不起作用。
As we are developing an application, we change fields, add columns in the database, or remove obsolete ones. Different teams tackle this in different ways. Some teams re-create the database with each new build. This works for new applications, because there is no existing data. However, after an application exists in production and has associated data, this approach won’t work.
应用程序需要考虑作为产品一部分的数据。与敏捷开发中的许多事情一样,需要数据库专家、程序员和测试人员在团队中共同努力,以确保成功发布数据库更改。Janet 看到了处理数据转换和向后兼容性的几种不同策略。当团队进行更改时,开发人员或数据库管理员可以创建数据库脚本。这些脚本成为构建的一部分,并不断进行测试。另一种选择是团队在完成所有数据库更改后对数据库运行“差异”。
An application needs to consider the data that is part of the product. As with so much in agile development, a joint effort by database experts, programmers, and testers on the team is required to ensure successful release of database changes. Janet has seen a couple of different tactics for dealing with data conversion and backward compatibility. Database scripts can be created by the developers or database administrators as the team makes changes. These scripts become part of the build and are continually tested. Another option is for the team to run “diffs” on the database after all of the database changes have been made.
如果您是测试人员,请让您的数据库管理员/开发人员帮助您的团队确保模式在生产、测试和暂存环境中保持一致。找到一种方法来保证在测试环境中所做的所有更改都将在发布期间在暂存和生产环境中完成。保持架构在列名、触发器、约束、索引和其他组件方面匹配(除了仍在开发中的新更改)。适用于编码和测试的同一学科也应适用于数据库开发和维护。
If you’re a tester, ask your database administrator/developer to help your team ensure that schemas are kept consistent among the production, testing, and staging environments. Find a way to guarantee that all changes made in the test environments will be done in the staging and production environments during release. Keep the schemas matching (except for the new changes still under development) in terms of column names, triggers, constraints, indices, and other components. The same discipline applied to coding and testing also should be applied to database development and maintenance.
自动化数据迁移可增强您测试它们的能力并减少人为错误的可能性。SQL、存储过程等原生数据库工具、SQL*Loader 和 bcp 等数据导入工具、shell 脚本和 Windows 命令文件可用于自动化,因为它们可以轻松克隆和更改。
Automating data migrations enhances your ability to test them and reduces the chance for human error. Native database tools such as SQL, stored procedures, data import tools such as SQL*Loader and bcp, shell scripts, and Windows command files can be used for automation because they can be cloned and altered easily.
无论数据库更新和转换脚本是如何创建或维护的,都需要对其进行测试。确保在更新脚本中捕获所有更改的最佳方法之一是使用可用的客户数据。客户习惯于以奇怪而美妙的方式使用应用程序,数据并不总是像我们希望的那样干净。如果开发团队清理数据库并对列施加额外限制,一旦查询触及与新限制不匹配的数据,客户站点上的应用程序可能会崩溃。您需要确保所做的任何更改仍与现有数据兼容。
No matter how the database update and conversion scripts are created or maintained, they need to be tested. One of the best ways to ensure all of the changes have been captured in the update scripts is to use the customer’s data if it is available. Customers have a habit of using the application in weird and wonderful ways, and the data is not always as clean as we would like it. If the development team cleans up the database and puts extra restrictions on a column, the application on the customer’s site might blow up as soon as a query touches a piece of data that does not match the new restrictions. You need to make sure that any changes you’ve made are still compatible with existing data.
在规划数据转换时,将数据清理视为缓解策略的一部分。您有机会获取以我们之前提到的一些“奇怪而美妙”的方式输入的数据,并对其进行修改或操作,使其符合新的约束条件。这种类型的工作可能需要很长时间才能完成,但在维护数据完整性方面通常非常值得。
When planning a data conversion, think about data cleanup as part of the mitigation strategy. You have the opportunity to take the data that was entered in some of the “weird and wonderful” ways we mentioned before and massage or manipulate it so it conforms to the new constraints. This type of job can take a long time to do but is often very worthwhile in terms of maintaining data integrity.
不是每个人都能在登台环境中对生产数据进行足够好的模拟。如果客户的数据不可用,缓解策略是在客户站点安装 UAT。另一种降低风险的方法是尽量避免大规模更新并在较小的阶段发布。在开发旧功能的同时开发新功能,并使用系统属性来“开启”其中一个功能。旧功能可以继续在生产中工作,直到新功能完成。同时,可以在每次迭代时对新代码进行测试。可以在不影响旧代码的情况下将新的列和表添加到生产表中,从而最大限度地减少最终发布的数据迁移或转换。
Not everyone can do a good enough simulation of production data in the staging environment. If a customer’s data is not available, a mitigation strategy is to have a UAT at the customer site. Another way to mitigate risk is to try to avoid large-scale updates and release in smaller stages. Develop new functionality in parallel with the old functionality and use a system property to “turn on” one or the other. The old functionality can continue to work in production until the new functionality is complete. Meanwhile, testing can be done on the new code at each iteration. New columns and tables can be added to production tables without affecting the old code so that the data migration or conversion for the final release is minimized.
组织通常有一个单独的团队部署到生产或创建产品集。这些团队成员应该有机会完全按照生产环境进行部署。如果他们使用部署到暂存作为他们的试验场,他们可以在向客户发布之前解决任何问题。
Organizations often have a separate team that deploys to production or creates the product set. These team members should have the opportunity to practice the deployment exactly as they would for production. If they use the deployment to staging as their proving ground, they can work out any of the problems long before they release to the customer.
测试产品安装还可以意味着测试不同操作系统或硬件的收缩包装产品的各种安装。产品表现如何?它能达到预期的效果吗?系统需要关闭多长时间才能安装?我们可以在不中断的情况下进行部署吗?我们能否让用户体验尽可能愉快?
Testing product installations can also mean testing various installations of shrink-wrapped products to different operating systems or hardware. How does the product behave? Does it do what is expected? How long will the system need to be down for installation? Can we deploy without taking an outage? Can we make the user experience as pleasant as possible?
花点时间确定测试安装的要求。如果你让你的客户满意,那最终是值得的。
Take the time you need to determine what your requirements are for testing installation. It will be worth it in the end if you satisfy your customers.
不同开发团队成员之间的持续沟通始终很重要,但在我们完成发布时尤为重要。如果需要,召开额外的站立会议,以确保一切都为发布做好准备。如果有可能忘记某些步骤,请为发布任务写卡片。
Constant communication between different development team members is always important, but it’s especially critical as we wrap up the release. Have extra stand-up meetings, if needed, to make sure everything is ready for the release. Write cards for release tasks if there’s any chance some step might be forgotten.
任务提醒,无论是在完整的实施计划中,还是像 Lisa 的团队那样只是写在任务卡上,通常都是必要的。在简单的实现中,白板效果很好。
Reminders of tasks, whether they are in a full implementation plan or just written on task cards as Lisa’s team does, are often necessary. On simple implementations, a whiteboard works well.
通过以多种形式不断跟踪进度,例如构建、回归测试套件、故事板和燃尽图,团队通常可以在发布遇到麻烦时提前知道。有时间放下故事并重新调整。尽管如此,最后一刻的灾难还是可能发生。如果构建机器在迭代的最后一天坏了怎么办?如果测试数据库崩溃导致最终测试无法完成怎么办?如果直到最后的功能测试才检测到 showstopper 错误怎么办?
By constantly tracking progress in many forms, such as builds, regression test suites, story boards, and burndown charts, a team usually knows well in advance when it’s in trouble on a release. There’s time to drop stories and readjust. Still, last-minute disasters can happen. What if the build machine breaks on the last day of the iteration? What if the test database crashes so that final testing can’t be completed? What if a showstopper bug isn’t detected until final functional testing?
我们强烈建议不要在迭代中增加额外的天数,因为它会占用下一个迭代或发布开发。一个经验丰富的团队可能足够灵活,可以做到这一点,但它可能会破坏一个新团队。然而,危急时刻需要采取危急措施。如果您每两周发布一次,您可以简单地跳过实际发布,将时间安排到下一次迭代中以更正问题并完成,然后在下一个预定日期发布。如果测试任务被推迟或忽略而发布继续进行,请向团队提出这个问题。测试需求是否发生变化,或者团队是否在冒险并牺牲质量以满足最后期限?如果交付日期已确定且处于危险之中,团队应缩减发布范围。
We strongly advise against adding extra days to an iteration, because it will eat into the next iteration or release development. An experienced team might be flexible enough to do this, but it can derail a new team. Still, desperate times call for desperate measures. If you release every two weeks, you may simply be able to skip doing the actual release, budget time into the next iteration to correct the problems and finish up, and release on the next scheduled date. If testing tasks are being put off or ignored and the release goes ahead, bring up this issue with the team. Did the testing needs change, or is the team taking a chance and sacrificing quality to meet a deadline? The team should cut the release scope if the delivery date is fixed and in jeopardy.
如果您的发布周期更长,更像是三个月,您应该提前知道您的发布是否处于危险之中。您可能已经计划了至少两周的结束游戏,这将仅用于最终验证。当您有一个较长的发布周期时,您就有更多的时间来确定您应该做什么,无论是删除功能还是更改时间表。
If your release cycle is longer, more like three months, you should know in advance if your release is in jeopardy. You probably have planned an end game of at least two weeks, which will just be for final validation. When you have a longer release cycle, you have more time to determine what you should do, whether it’s dropping functionality or changing the schedule.
如果您的组织需要在固定的日期发布某些功能,并且最后一刻的故障威胁到发布,请评估您的备选方案。看看您是否可以继续相同的开发周期,但将发布本身推迟一天或一周。也许有问题的代码可以暂时退出,稍后再打补丁。客户对什么对企业有用有最终决定权。
If your organization requires certain functionality to be released on a fixed day and last-minute glitches threaten the release, evaluate your alternatives. See if you can continue on your same development cycle but delay the release itself for a day or a week. Maybe the offending piece of code can be backed out temporarily and a patch done later. The customers have the ultimate say in what will work for the business.
通过良好的规划、密切协作、通过测试驱动编码以及在编码时进行测试,努力防止出现“不行”的情况。如果您的跟踪显示该版本可能处于危险之中,请尽可能删除无法完成的功能。如果发生不好的和意想不到的事情,不要惊慌。让整个团队和客户团队参与进来,集思广益,找出最佳解决方案。
Work to prevent a “no go” situation with good planning, close collaboration, driving coding with tests, and testing as you code. If your tracking shows the release could be in jeopardy, remove the functionality that can’t be finished, if possible. If something bad and unexpected happens, don’t panic. Involve the whole team and the customer team, and brainstorm about the best solution.
有几种不同的方式可以让您的客户参与进来以获得他们的认可或反馈。用户验收测试可以相当正式,需要企业签字。它表示接受发布。Alpha 或 Beta 测试是一种获取有关您要发布但尚未准备就绪的产品的反馈的方法。
There are a couple of different ways in which to involve your customers to get their approval or feedback. User Acceptance Testing can be fairly formal, with sign-offs from the business. It signifies acceptance of a release. Alpha or beta testing is a way to get feedback on a product you are looking to release but which is not quite ready.
用户验收测试 (UAT) 在大型定制应用程序和内部应用程序中都很重要。它由所有受影响的业务部门执行,以验证系统的可用性并确认系统的现有和新的(强调新的)业务功能。你的客户是那些必须使用该应用程序的人,因此他们需要确保它可以在他们的系统和数据上正常工作。
User Acceptance Testing (UAT) is important in large customized applications as well as internal applications. It’s performed by all affected business departments to verify usability of the system and to confirm existing and new (emphasis on new) business functionality of the system. Your customers are the ones who have to live with the application, so they need to make sure it works on their system and with their data.
在前面的章节中,我们经常谈到让客户尽早参与,但在那个时候,测试是针对正在开发的特定功能进行的。UAT 通常在团队决定质量足以发布后完成。但有时,时间表决定了发布周期。如果是这种情况,请尝试将 UAT 周期向上移动以与您的最终游戏并行运行。应用程序应该足够稳定,以便您的团队可以在部署到暂存系统的同时部署到客户的测试系统。
In previous chapters we’ve often talked about getting the customers involved early, but at those times, the testing is done on specific features under development. UAT is usually done after the team decides the quality is good enough to release. Sometimes though, the timeline dictates the release cycle. If that is the case, then try moving the UAT cycle up to run parallel with your end game. The application should be stable enough so that your team could deploy to the customer’s test system at the same time as they deploy to staging.
图 20-1显示了在发布周期结束时具有正常 UAT 的示例时间线。团队开始着手下一个版本,进行发布计划,并在所有团队成员准备就绪的情况下开始第一次迭代。
Figure 20-1 shows an example timeline with a normal UAT at the end of the release cycle. The team starts working on the next release, doing release planning, and starts the first iteration with all team members ready to go.
与客户合作,让他们了解流程、他们的角色以及对他们的期望。如果 UAT 不顺利,则可能需要高水平的支持。一个有经验的客户测试团队可能已经定义了测试用例,但大多数时候它的测试是临时的。客户可能会像对待日常工作一样对待他们的测试,但可能会专注于新功能。这是一个观察人们如何使用该系统并从他们那里获得有关哪些方面运作良好以及哪些改进对他们有帮助的反馈。
Work with customers so that they understand the process, their role, and what is expected of them. If the UAT is not smooth, then the chances are there will be a high level of support needed. An experienced customer test team may have defined test cases, but most often its testing is ad hoc. Customers may approach their testing as if they were doing their daily job but will probably focus on the new functionality. This is an opportunity to observe how people use the system and to get feedback from them on what works well and what improvements would help them.
测试人员可以通过审查测试运行和记录的缺陷,并通过跟踪缺陷直至完成,为执行 UAT 的客户提供支持。我们都发现向参与 UAT 的客户提供开发期间完成的所有测试的报告以及结果很有帮助。这有助于他们决定将自己的测试集中在哪里。
Testers can provide support to the customers who are doing the UAT by reviewing tests run and defects logged, and by tracking defects to completion. Both of us have found it helpful to provide customers involved in doing UAT with a report of all of the testing done during development, along with the results. That helps them decide where to focus their own testing.
如果您是一个向大型客户群分发软件的组织,您可能没有正式的 UAT。您更有可能加入 alpha 或 beta 测试。您的团队希望从您的真实客户那里获得对新功能的反馈,这是一种这样做的机制。Alpha 测试是软件新版本的早期分发。由于可能存在一些重大错误,因此您需要明智地挑选客户。如果您选择这种客户反馈方法,请确保您的客户了解他们的角色。Alpha 测试是为了获得对功能的反馈——而不是报告错误。
If you are an organization that distributes software to a large customer base, you may not have a formal UAT. You are much more likely to incorporate alpha or beta testing. Your team will want to get feedback on new features from your real customers, and this is one mechanism for doing so. Alpha testing is early distribution of new versions of software. Because there are likely to be some major bugs, you need to pick your customers wisely. If you choose this method of customer feedback, make sure your customers understand their role. Alpha testing is to get feedback on the features—not to report bugs.
Beta 测试更接近 UAT。预计该版本相当稳定,可以实际使用。对于大多数客户来说,它可能还没有“准备好迎接黄金时段”,但许多客户可能觉得新功能值得冒险。顾客应该了解这不是正式版本,您要求他们测试您的产品并报告错误。
Beta testing is closer to UAT. It is expected that the release is fairly stable and can actually be used. It may not be “ready for prime time” for most customers, but many customers may feel the new features are worth the risk. Customers should understand that it is not a formal release and that you are asking them to test your product and report bugs.
作为测试人员,了解客户如何看待产品很重要,因为它可能会影响您的测试方式。Alpha 和 Beta 测试可能是您与最终用户互动的唯一机会,因此请利用这个机会了解产品如何满足他们的需求。
As a tester, it is important to understand how customers view the product, because it may affect how you test. Alpha and beta testing may be the only time you get to interact with end users, so take advantage of the chance to learn how well the product meets their needs.
如果您在大型组织工作或正在开发大型复杂系统的组件,您可能需要在开发完成后安排时间进行测试。有时 UAT 测试或测试协调并不像预期的那样顺利,因此时间线会延长。包含所有生产系统的测试版本的测试环境可能仅在较小的预定时间窗口内可用。您可能需要与处理与您交互的其他应用程序的团队协调测试会话。不管是什么原因,您都需要额外的测试时间,其中不包括整个开发团队。
If you work in a large organization or are developing a component of a large, complex system, you may need to budget time for testing after development is complete. Sometimes the UAT testing, or the test coordination, isn’t as smooth as it could be, so the timeline stretches out. Test environments that include test versions of all production systems may only be available for small, scheduled windows of time. You may need to coordinate test sessions with teams working on other applications that interact with yours. Whatever the reason, you need extra testing time that does not include the whole development team.
图 20-2显示了扩展 UAT 的时间线示例。发生这种情况的原因有很多,而且问题可能并不总是出在 UAT 上。大多数团队都准备好开始下一个版本的工作,但通常测试人员仍在与客户合作,完成最终测试。有时,程序员也会参与其中。有几个选项。如果团队足够大,您可以开始下一个版本,而几个团队成员使用现有版本(版本 2—图 20-2中的备选方案 2 )。如果您有一个小团队,您可能需要考虑迭代 0,让程序员在新功能上进行重构或尖峰(实验),这样与客户一起工作的测试人员就不会落后(版本 2—图 20 中的备选方案1- 2 ).
Figure 20-2 shows an example timeline where the UAT was extended. This could happen for any number of reasons, and the issue may not always be UAT. Most of the team is ready to start working on the next release, but often a tester is still working with customers, completing final testing. Sometimes a programmer will be involved as well. There are a couple of options. If the team is large enough, you can probably start the next release while a couple of team members work with the existing release (Release 2—Alternative 2 in Figure 20-2). If you have a small team, you may need to consider an Iteration 0 with programmers doing refactoring or spikes (experiments) on new functionality so that the tester working with the customer does not get left behind (Release 2—Alternative 1 in Figure 20-2).
创造性地处理项目现实强加给你的团队的情况。虽然计划很少会按预期进行,但提前计划仍然可以帮助您确保合适的人员到位以及时交付产品。
Be creative in dealing with circumstances imposed on your team by the realities of your project. While plans rarely work as expected, planning ahead can still help you make sure the right people are in place to deliver the product in a timely manner.
在本章的第一节中,我们讨论了产品的构成要素。这个问题的答案实际上取决于听众:谁在接受产品,他们的期望是什么?
In the first section of this chapter we talked about what makes a product. The answer to this will actually depend on the audience: Who is accepting the product, and what are their expectations?
如果您的客户需要满足 SOX (Sarbanes-Oxley) 合规性要求,则需要某些可交付成果。例如,与 Janet 合作的一位客户认为测试结果应该完整记录,并将测试结果作为他们的 SOX 合规性测量点之一,而另一位客户根本没有测量测试结果。与合规和审计人员合作,在您开始项目时确定报告需求。
If your customers need to meet SOX (Sarbanes-Oxley)compliance requirements, there will be certain deliverables that are required. For example, one customer Janet has worked with felt test results should be thoroughly documented, and made test results one of their SOX compliance measurement points, while a different customer didn’t measure test results at all. Work with compliance and audit personnel to identify reporting needs as you begin a project.
多少文档就足够了?珍妮特在回答这个问题之前总是问两个问题:“它是给谁的?” 和“他们用它做什么?” 如果这些问题没有充分的答案,那么请考虑是否真的需要该文档。
How much documentation is enough? Janet always asks two questions before answering that question: “Who is it for?” and “What are they using it for?” If there are no adequate answers to those questions, then consider whether the documentation is really needed.
可交付成果并不总是针对最终客户的,也不总是以软件的形式出现。有很多内部客户,例如生产支持团队成员。他们需要什么来使他们的工作更轻松?工作流程图可以帮助他们理解新功能。他们可能想知道是否有适当的变通办法,以便他们可以帮助客户解决问题。
Deliverables are not always for the end customer, and they aren’t always in the form of software. There are many internal customers, such as the production support team members. What will they need to make their job easier? Workflow diagrams can help them understand new features. They would probably like to know if there are work-arounds in place so they can help customers through problems.
Janet 经常被问及代码的测试覆盖率,通常是管理层问的。单元测试或回归测试正在测试多少应用程序?问题是这个数字本身只是一个数字,它可能高或低的原因有很多。此外,代码覆盖率不会告诉您可能遗漏的功能,因为这些功能尚不存在代码。可交付成果(例如代码覆盖率)的受众不应该是管理层,而应该是团队本身。它可以用来查看代码的哪些区域没有被测试。
Janet often gets asked about test coverage of code, usually by management. How much of the application is being tested by the unit tests or regression tests? The problem is that the number by itself is just a number, and there are so many reasons why it might be high or low. Also, code coverage doesn’t tell you about features that might have been missed, for which no code exists yet. The audience for a deliverable such as code coverage should not be management, but the team itself. It can be used to see what areas of the code are not being tested.
培训也可以被视为一种可交付成果。许多应用程序需要为客户定制培训课程。其他人可能只需要在线帮助或用户手册。培训可以决定您的产品是否成功,因此考虑这一点很重要。Lisa 的团队经常为测试人员或产品负责人编写任务卡,以确保安排好培训材料和课程。有些人可能觉得培训不是测试人员或开发团队中任何其他人的工作。然而,敏捷团队的目标是与业务尽可能紧密地合作。测试人员通常拥有领域专业知识,至少能够确定新功能或更新功能可能需要的培训。即使培训不是测试人员的责任,如果企业没有计划培训课程,她也可以提出问题。
Training could be considered a deliverable as well. Many applications require customized training sessions for customers. Others may only need online help or a user manual. Training could determine the success of your product, so it’s important to consider. Lisa’s team often writes task cards for either a tester or the product owner to make sure training materials and sessions are arranged. Some people may feel training isn’t the job of testers or anyone else on the development team. However, agile teams aim to work as closely as possible with the business. Testers often have the domain expertise to be able to at least identify training that might be needed for new or updated features. Even if training isn’t the tester’s responsibility, she can raise the issue if the business isn’t planning training sessions.
许多敏捷团队都有技术作家作为团队的一部分,负责编写在线帮助或电子形式的文档。甚至包括一个应用程序帮助入门的培训视频,团队的不同成员是培训师。创造一个成功的产品是团队的责任。
Many agile teams have technical writers as part of the team that write online help or electronic forms of documentation. One application even included training videos to help get started, and different members of the team were the trainers. It is the responsibility of the team to create a successful product.
当我们谈论发布产品时,我们的意思是让客户可以使用任何可能采用的格式。您的组织可能有一个不断更新的网站或一个交付给少数大客户的自定义应用程序。也许该产品采用收缩包装并交付给全球数百万台 PC,或者从 Internet 下载。
When we talk about releasing the product, we mean making it available to the customer in whatever format that may take. Your organization might have a website that gets updated or a custom application that is delivered to a few large customers. Maybe the product is shrink-wrapped and delivered to millions of PCs around the world, or downloaded off the Internet.
你怎么知道什么时候完成?验收标准是定义何时接受产品的传统方式。绩效标准可能有被满足。我们在每次迭代开始时为每个故事捕获这些,我们也可以在开始主题或史诗时为更大的功能集指定它们。客户可以设置质量标准,例如自动测试覆盖一定百分比的代码,或者某些测试必须通过。零严重错误或对系统有严重影响的零错误等行项目通常是发布标准的一部分。客户需要决定他们如何知道产品何时具有足够的价值。测试人员可以帮助他们定义实现目标的发布标准。
How do you know when you’re done? Acceptance criteria are a traditional way of defining when to accept the product. Performance criteria may have to be met. We capture these for each story at the start of each iteration, and we may also specify them for larger feature sets when we begin a theme or epic. Customers may set quality criteria such as a certain percentage of code covered by automated tests, or that certain tests must pass. Line items such as having zero critical bugs, or zero bugs with serious impact to the system, are often part of the release criteria. The customers need to decide how they’ll know when there’s enough value in the product. Testers can help them define release criteria that accomplish their goals.
敏捷团队努力实现质量目标的精神,而不仅仅是文字。他们不会将错误的严重性降级为中等,因此他们可以说他们达到了没有高严重性错误的标准。相反,他们经常查看错误趋势并想办法确保高严重性错误不会在生产中出现。
Agile teams work to attain the spirit of the quality goals, not just the letter. They don’t downgrade the severity of bugs to medium so they can say they achieved the criterion of no high-severity bugs. Instead, they frequently look at bug trends and think of ways to ensure that high-severity bugs don’t occur in production.
您应该预先与客户协商您的质量水平,以免出现不愉快的意外。您的团队和客户使用真实示例定义的验收测试应作为发布进度的里程碑。如果您的客户对错误的容忍度非常低,并且 100% 的验收测试必须通过,那么您的迭代速度应该考虑到这一点。如果新功能比错误修复更重要,那么,也许你会在发布时出现错误。
Your quality level should be negotiated with your customer up front so that there are no unpleasant surprises. The acceptance tests your team and your customers defined, using real examples, should serve as milestones for progress toward release. If your customer has a very low tolerance for bugs, and 100% of those acceptance tests must be passing, your iteration velocity should take that into consideration. If new features are more important than bug fixes, well, maybe you will be shipping with bugs.
每个项目、每个团队、每个业务都是独一无二的。敏捷团队与业务专家合作,决定何时准备好将软件交付生产。如果发布截止日期一成不变,企业将不得不修改范围。如果在软件具有足够的价值时有足够的灵活性来发布,则团队可以决定何时满足质量标准并且软件可以投入生产。
Each project, each team, each business is unique. Agile teams work with the business experts to decide when they’re ready to deliver software to production. If the release deadline is set in stone, the business will have to modify scope. If there’s enough flexibility to release when the software has enough value, the teams can decide when the quality criteria have been met and the software can go to production.
传统软件开发的工作时间很长,截止日期提前很多,从一个阶段到下一个阶段需要清除障碍。敏捷开发让我们能够以小增量生产高质量的软件,并在必要时发布。开发和客户团队可以密切合作来定义和决定发布什么以及何时发布。测试人员可以在此目标设定过程中发挥关键作用。
Traditional software development works in long time frames, with deadlines set far in advance and hurdles to clear from one phase to the next. Agile development lets us produce quality software in small increments and release as necessary. The development and customer teams can work closely to define and decide what to release and when. Testers can play a critical role in this goal-setting process.
许多组织都有发布管理团队,但如果您没有,仍然会有人做这项工作。在小型组织中,很多时候都是 QA 经理担任此角色。负责发布的人员可以与利益相关者召开发布准备会议以评估准备情况。
Many organizations have a release management team, but if you don’t, someone still does the work. Many times in a small organization it is the QA manager who fulfills this role. The person leading the release may hold a release readiness meeting with the stakeholders to evaluate readiness.
发布准备清单是一个很好的工具,可以用来检查对你的团队来说重要的事情。此清单的目的是帮助团队客观地确定已完成的内容并确定与未完成任务相关的风险。
A release readiness checklist is a great tool to use to walk through what is important to your team. The intention of this checklist is to help the team objectively determine what was completed and identify the risks associated with not completing a task.
例如,如果不需要培训因为对产品所做的更改对最终用户是透明的,那么风险就很低。但是,如果在系统中创建新用户的过程发生重大变化,生产支持或帮助团队面临的风险将非常高,并且可能需要延迟。必须考虑所有利益相关者的需求。
For example, if training is not required because the changes made to the product were transparent to the end user, then the risk is low. However, if there were significant changes to the process for how a new user is created in the system, the risk would be very high to the production support or help teams, and may warrant a delay. The needs of all stakeholders must be considered.
版本说明对于任何产品版本都很重要。这些的形式取决于听众。如果您的产品是针对开发人员的,那么“read me”文本文件可能就可以了。在其他情况下,您可能想让它们更正式。无论是何种媒体,它们都应该满足受众的需求。不要提供很多不需要的附加信息。
Release notes are important for any product release. The formality of these depends on the audience. If your product is aimed at developers, then a “read me” text file is probably fine. In other cases, you may want to make them more formal. Whatever the media, they should address the needs of the audience. Don’t provide a lot of added information that isn’t needed.
当 Janet 获得新版本时,她做的第一件事就是检查版本和所有组件。“我得到了他们所说的给我的东西吗?安装前是否需要考虑特殊说明,例如依赖项或升级脚本?” 这些是发行说明中需要回答的简单问题。其他要包括的内容是客户应该寻找的新功能。
When Janet gets a new release, one of the first things she does is check the version and all of the components. “Did I get what they said they gave me? Are there special instructions I need to consider before installing, such as dependencies or upgrade scripts?” Those are good simple questions to answer in release notes. Other things to include are the new features that the customer should look for.
发行说明应特别考虑不属于您的开发团队交付内容的组件,例如由不同团队准备的帮助文件或用户手册。有时旧的发行说明会留在发行媒体上,这对最终用户可能有用也可能没用。考虑什么适合您的团队和您的应用程序。
Release notes should give special consideration to components that aren’t part of what your development team delivered, such as a help file or user manuals prepared by a different team. Sometimes old release notes get left on the release media, which may or may not be useful to the end user. Consider what is right for your team and your application.
我们已经就持续集成谈了很多。我们倾向于认为这是理所当然的,而忘记了良好的配置管理意味着什么。“一次构建,多次部署”是让我们在发布时充满信心的部分原因。我们知道我们在暂存中测试的构建与客户测试的构建相同在 UAT 中,是我们发布到生产环境的构建。这对于成功发布至关重要。
We’ve talked a lot about continual integration. We tend to take it for granted and forget what good configuration management means. “Build once, deploy multiple times” is part of what gives us confidence when we release. We know that the build we tested in staging is the same build that the customer tested in UAT and is the build we release to production. This is critical for a successful release.
如果该产品是为外部客户准备的,那么安装应该很容易,因为安装可能是客户首先看到的产品。了解您的听众及其对错误的容忍度。产品将如何交付?例如,如果要从 Internet 下载它,那么它应该是一个简单的下载和安装。如果它是一个庞大的企业系统,那么您的组织可能需要派一名技术支持人员来帮助安装该产品。
If the product is intended for an external customer, the installation should be easy, because the installation may be the first look at the product that customer has. Know your audience and its tolerance level for errors. How will the product be delivered? For example, if it is to be downloaded off the Internet, then it should be a simple download and install. If it is a huge enterprise system, then maybe your organization needs to send a support person with the product to help with the install.
在我们向客户推出新软件之前,我们最好确定他们已经准备好使用它。我们必须确保他们知道期望的新功能是什么,并且他们有一些方法来处理出现的问题。
Before we spring new software on our customers, we’d better be certain they are ready for it. We must be sure they know what new functionality to expect and that they have some means to deal with problems that arise.
许多组织都有一个生产或运营支持团队,负责维护代码并在代码投入生产后为客户提供支持。如果您的公司有生产支持团队,那么该团队就是您的第一个客户。让它成为你的伙伴。生产支持团队收到来自客户的缺陷报告和增强请求,他们可以与您的团队一起识别高风险区域。
Many organizations have a production or operations support team that maintains the code and supports customers after it’s in production. If your company has a production support team, that group is your first customer. Make it your partner as well. Production support teams receive defect reports and enhancement requests from the customers, and they can work with your team to identify high-risk areas.
很多时候,生产支持团队是接受开发团队发布的团队。如果您的组织有这种类型的交接,那么您的开发团队与生产支持团队密切合作以使其平稳过渡非常重要。确保生产支持团队了解如何使用系统的日志文件以及消息和监控系统,以便跟踪操作并快速识别问题。
Very often the production support team is the team that accepts the release from the development team. If your organization has this type of hand-off, it is important that your development team works closely with the production support team to make it a smooth transition. Make sure the production support team understands how to use the system’s log files and the messaging and monitoring systems in order to keep track of operations and identify problems quickly.
每次部署到生产需要中断时,您的客户就无法使用该产品。如果您的产品是网站,这可能会产生巨大的影响。如果您的产品是要下载到 PC 上的独立产品,则影响很小。敏捷团队经常发布以最大化业务价值,小版本发布产生大负面影响的风险较低。与企业合作以按时间段发布时间是常识最大限度地减少中断。尽可能自动化和简化部署流程,以减少停机时间。快速部署过程在短迭代开发期间也很有帮助,我们可能会在一天内部署十几次。
Every time a deployment to production requires an outage, the product is unavailable to your customer. If your product is a website, this may be a huge impact. If your product is an independent product to be downloaded onto a PC, the impact is low. Agile teams release frequently to maximize value to the business, and small releases have a lower risk of a large negative impact. It’s common sense to work with the business to time releases for time periods that minimize disruption. Automate and streamline deployment processes as much as possible to keep downtime windows small. A quick deployment process is also helpful during development in short iterations where we may deploy a dozen times in one day.
新版本应该对客户尽可能透明。发布后需要的紧急发布或补丁越少,您的客户对产品和开发团队的信心就越大。
New releases should be as transparent as possible to the customer. The fewer emergency releases or patches required after a release, the more confidence your customer will have in both the product and the development team.
从每次发布中吸取教训并采取行动使下一次发布更加顺利。获取参与规划的所有角色,例如系统和数据库管理员。评估每个版本并想办法改进下一个版本。
Learn from each release and take actions to make the next one go more smoothly. Get all roles, such as system and database administrators, involved in the planning. Evaluate each release and think of ways to improve the next one.
本章涵盖了以下几点:
This chapter covered the following points:
产品的成功交付不仅仅包括您正在构建的应用程序。规划非软件可交付成果,例如文档、法律声明和培训。
Successful delivery of a product includes more than just the application you are building. Plan the non-software deliverables such as documentation, legal notices, and training.
最后的游戏是一个机会,可以对你的产品进行最后的润色。
The end game is an opportunity to put the spit and polish, the final finishing touches, on your product.
其他小组可能负责最终游戏和发布的环境、工具和其他组件。提前与他们协调。
Other groups may be responsible for environments, tools, and other components of the end game and release. Coordinate with them ahead of time.
请务必测试数据库更新脚本、数据转换和安装的其他部分。
Be sure to test database update scripts, data conversions, and other parts of the installation.
UAT 为客户提供了测试其数据并建立对产品信心的机会。
UAT is an opportunity for customers to test against their data and to build their confidence in the product.
根据需要为额外的周期预算时间,例如与外部各方协调测试的后期开发周期。
Budget time for extra cycles as needed, such as post-development cycles to coordinate testing with outside parties.
在发布计划期间建立发布验收标准,以便您可以知道何时准备好发布。
Establish release acceptance criteria during release planning so that you can know when you’re ready to release.
测试人员通常参与管理发布和测试包装。
Testers often are involved in managing releases and testing the packaging.
发布产品时,要考虑整个包装——客户的需求和期望。
When releasing the product, consider the whole package—what the customer needs and expects.
从每个版本中学习,并进行调整以改进您的流程。
Learn from each release, and adapt to improve your processes.
在第 21 章“关键成功因素”中,我们将所有内容放在一起并总结了敏捷测试方法。
In Chapter 21, “Key Success Factors,” we pull things together and summarize the agile approach to testing.
在经历了一个迭代之后,随着敏捷测试人员参与许多活动,我们现在可以挑选出一些关键因素,这些因素可以帮助测试人员在敏捷团队中取得成功,并帮助敏捷团队成功交付高质量的产品。我们认为敏捷测试人员可以提供一些特别的东西。“感染敏捷”的测试人员学习如何应用敏捷实践和原则来帮助他们的整个团队生产出更好的产品。敏捷团队中“被测试感染”的程序员学习如何使用测试来产生更好的工作。角色之间的界限很模糊,但这是一件好事。每个人都专注于质量。
Having traveled through an iteration and beyond, following an agile tester as she engages in many activities, we can now pick out some key factors that help testers succeed on agile teams and help agile teams succeed at delivering a high-quality product. We think agile testers have something special to offer. “Agile-infected” testers learn how to apply agile practices and principles to help their whole team produce a better product. “Test-infected” programmers on agile teams learn how to use testing to produce better work. Lines between roles are blurred, but that’s a good thing. Everyone is focused on quality.
我们从自己的反复试验以及与我们合作的团队中收集了一些适用于敏捷团队和测试人员的关键测试指南。这些指南建立在敏捷测试矩阵、我们学习克服文化和组织障碍的经验、我们在敏捷团队中扮演测试人员角色的冒险经历以及我们弄清楚如何最好地使用测试自动化的经验之上。我们喜欢幸运数字,因此在本章中我们将介绍七个帮助敏捷测试人员取得成功的关键因素。
We have gleaned some critical testing guidelines for agile teams and testers from our own trial and error as well as from teams with which we’ve worked. These guidelines are built on the agile testing matrix, on our experience of learning to overcome cultural and organizational obstacles, our adventures in performing the tester role on agile teams, and our experience of figuring out how best to use test automation. We like lucky numbers, so in this chapter we present seven key factors that help an agile tester succeed.
我们请了一小群正在审阅我们某些章节的人来建议介绍这些成功因素的顺序。结果差异很大,尽管许多人(但不是全部)同意前两名。选择能给您带来最大投资回报的成功因素,并从今天开始努力。
We asked a small group of people who were reviewing some of our chapters to suggest the order in which to present these success factors. The results varied quite a bit, although many (but not all) agreed on the top two. Pick the success factor that will give you the biggest return on investment, and start working on it today.
当整个开发团队负责测试和质量时,您将拥有各种各样的技能和经验水平来应对可能出现的任何测试问题。对于一群熟练的程序员来说,测试自动化不是一个大问题。当测试是团队的首要任务,并且任何人都可以报名参加测试任务时,团队就会设计可测试的代码。
When the whole development team takes responsibility for testing and quality, you have a large variety of skill sets and experience levels taking on whatever testing issues might arrive. Test automation isn’t a big problem to a group of skilled programmers. When testing is a team priority, and anyone can sign up for testing tasks, the team designs testable code.
让测试人员真正成为开发团队的一部分意味着为他们提供适应敏捷开发的快节奏所需的支持和培训。他们有时间学习新技能,以便与开发团队和客户团队的成员密切合作。
Making testers truly part of the development team means giving them the support and training they need to adapt to the fast pace of agile development. They have time to acquire new skills in order to collaborate closely with members of both the development and customer teams.
如果您管理一个敏捷团队,请使用第 II 部分“组织挑战”中的建议来帮助您的团队采用全团队方法。请记住,质量而非速度是敏捷开发的目标。您的团队需要测试人员帮助客户阐明需求,将需求转化为指导开发的测试,并提供独特的观点来促进可靠产品的交付。确保测试人员可以将他们的技能和专业知识传授给团队的其他成员。确保他们没有被束缚在诸如仅进行手动测试之类的角色中。确保当他们寻求帮助时(这可能需要他们相当大的勇气),他们的团队成员会给予帮助。反之亦然。只要有人需要他们可以提供的帮助,测试人员就应该挺身而出。
If you manage an agile team, use the suggestions in Part II, “Organizational Challenges,” to help your team adopt the whole-team approach. Remember that quality, not speed, is the goal of agile development. Your team needs testers to help customers clarify requirements, turn those into tests that guide development, and provide a unique viewpoint that will promote delivery of a solid product. Make sure the testers can transfer their skills and expertise to the rest of the team. Make sure they aren’t pigeonholed in a role such as only doing manual testing. Make sure that when they ask for help (which may require considerable courage on their part), their team members give it. The reverse is true, too; a tester should step up whenever someone needs assistance that they can provide.
如果您是敏捷团队的测试员,并且正在进行的计划会议和设计讨论中没有您,或者业务用户正在努力单独定义他们的故事和需求,那么是时候站起来和他们谈谈了团队的其他成员。与程序员坐在一起,邀请自己参加会议,并建议让测试人员、程序员和业务专家参与进来,尝试“三的力量”。有用,提供反馈并帮助客户提供示例。让你的问题成为团队的问题,让他们的问题成为你的问题。要求您的队友采用整体团队方法。
If you’re a tester on an agile team, and there are planning meetings and design discussions happening that don’t include you, or the business users are struggling to define their stories and requirements alone, it’s time to get up and go talk to the rest of the team. Sit with the programmers, invite yourself to meetings, and propose trying the “Power of Three” by involving a tester, a programmer, and a business expert. Be useful, giving feedback and helping the customers provide examples. Make your problems the team’s problems, and make their problems yours. Ask your teammates to adopt a whole-team approach.
请参阅第 2 章“敏捷测试人员的十项原则”,以了解“三的力量”如何运作的示例。
See Chapter 2, “Ten Principles for Agile Testers,” for an example of how the “Power of Three” works.
在第 2 章“敏捷测试人员的十项原则”中,我们告诫敏捷测试人员放弃他们可能带来的任何“质量警察”思维模式。你现在在一个敏捷团队中,程序员测试和测试人员尽他们所能帮助团队交付最好的产品。作为我们在第 2 章强调过,敏捷测试的态度是主动的、有创造力的、对新想法持开放态度并愿意承担任何任务。敏捷测试员不断磨练自己的手艺,随时准备协作,相信自己的直觉,并热衷于帮助团队和业务取得成功。
In Chapter 2, “Ten Principles for Agile Testers,” we cautioned agile testers to lose any “Quality Police” mind-set they might have brought with them. You’re on an agile team now, where programmers test and testers do whatever they can think of to help the team deliver the best possible product. As we emphasized in Chapter 2, an agile testing attitude is proactive, creative, open to new ideas, and willing to take on any task. The agile tester constantly hones her craft, is always ready to collaborate, trusts her instincts, and is passionate about helping the team and the business succeed.
我们并不是说你应该穿上你的超级测试员斗篷去保护世界免受错误的侵害。敏捷团队不允许大自负。您的队友分享您对质量的热情。专注于团队的目标,尽你所能帮助每个人把工作做到最好。
We don’t mean that you should put on your Super Tester cape and go protect the world from bugs. There’s no room for big egos on agile teams. Your teammates share your passion for quality. Focus on the team’s goals and do what you can to help everyone do their best work.
使用敏捷原则和价值观来指导您。始终尝试最简单的方法来满足测试需求。勇于寻求帮助和尝试新想法。专注于提供价值。尽可能直接和经常地沟通。灵活应对变化。请记住,敏捷开发是以人为中心的,我们都应该享受我们的工作。当有疑问时,回到价值观和原则来决定做什么。
Use agile principles and values to guide you. Always try the simplest approach to meeting a testing need. Be courageous in seeking help and experimenting with new ideas. Focus on delivering value. Communicate as directly and as often as possible. Be flexible in responding to change. Remember that agile development is people-centric, and that we should all enjoy our work. When in doubt, go back to the values and principles to decide what to do.
敏捷测试思维模式的一个重要组成部分是不断寻找更好的工作方式的动力。一个成功的敏捷测试员不断地磨练她的手艺。阅读好书、博客和文章以获得新的想法和技能。参加本地用户组会议。参与邮件列表讨论以获得问题或新想法的反馈。如果你的公司不花钱让你参加一个好的会议,那就把你学到的东西写成体验报告,换取免费的会议注册。回馈您的测试和敏捷开发社区也会对您有所帮助。
An important component of the agile testing mind-set is the drive to continually find better ways to work. A successful agile tester constantly polishes her craft. Read good books, blogs, and articles to get new ideas and skills. Attend local user group meetings. Participate in mailing list discussions to get feedback on problems or new ideas. If your company won’t pay for you to attend a good conference, put what you’ve learned into an experience report to exchange for a free conference registration. Giving back to your testing and agile development communities will help you, too.
有关敏捷测试思维的更多信息,请参阅第 2 章,“敏捷测试人员的十项原则”。
See Chapter 2, “Ten Principles for Agile Testers,” for more about the agile testing mindset.
尝试新的实践、工具和技术。鼓励您的团队尝试新方法。短迭代非常适合实验。你可能会失败,但它会很快,你可以尝试别的东西。
Experiment with new practices, tools, and techniques. Encourage your team to try new approaches. Short iterations are ideally suited to experimentation. You might fail, but it’ll be fast, and you can try something else.
如果您管理敏捷测试人员或敏捷团队,请给他们时间学习并为他们所需的培训提供支持。消除障碍,使他们能够做到最好。
If you manage agile testers or an agile team, give them time to learn and provide support for the training they need. Remove obstacles so that they can do their best work.
当您遇到影响测试的问题时,请将这些问题提交给团队。让团队集体讨论克服这些障碍的方法。回顾是讨论问题以及如何解决问题的地方。保持障碍积压并在每次迭代中解决一两个。使用大的可见图表,或它们的虚拟等效物,以确保每个人都知道出现的问题,并且每个人都可以跟踪编码和测试的进度。
When you’re faced with problems that impact testing, bring those problems to the team. Ask the team to brainstorm ways to overcome these obstacles. Retrospectives are one place to talk about issues and how to resolve them. Keep an impediment backlog and address one or two in every iteration. Use big visible charts, or their virtual equivalents, to ensure that everyone is aware of problems that arise and that everyone can track the progress of coding and testing.
如果没有测试自动化,敏捷团队能否成功?也许吧,但我们所知道的成功团队依赖于自动化回归测试。正如我们在本书中经常提到的,如果您将所有时间都花在手动回归测试上,您将永远没有时间进行重要的探索性测试,这些测试将找出代码中潜伏的破坏性行为。
Can an agile team succeed with no test automation? Maybe, but the successful teams that we know rely on automated regression tests. As we’ve said often in this book, if you’re spending all your time doing manual regression testing, you’ll never have time for the important exploratory testing that will ferret out the damaging behaviors lurking in the code.
敏捷开发使用测试来指导开发。为了编写代码使测试通过,您需要一种快速简便的方法来运行测试。如果没有套件提供的短反馈周期和安全网回归,您的团队很快就会陷入技术债务的泥潭,缺陷队列越来越多,速度越来越慢。
Agile development uses tests to guide development. In order to write code to make a test pass, you need a quick and easy way to run the test. Without the short feedback cycle and safety net regression that suites provide, your team will soon become mired in technical debt, with a growing defect queue and ever-slowing velocity.
自动化回归测试是一项团队工作。整个团队应该为每种类型的测试选择合适的工具。预先考虑测试将使程序员设计代码以便于测试自动化。使用敏捷测试象限和测试自动化金字塔来帮助您有效地自动化不同类型的测试。
Automating regression tests is a team effort. The whole team should choose appropriate tools for each type of test. Thinking about tests up front will let programmers design code for ease of test automation. Use the Agile Testing Quadrants and test automation pyramid to help you automate different types of tests effectively.
有关测试自动化金字塔的更多信息,请参见第 14 章“自动化策略”。
See Chapter 14, “Automation Strategy,” for more on the test automation pyramid.
记住从简单开始。您会惊讶于一些基本的自动化冒烟测试或自动化单元测试可以提供多少价值。
Remember to start simply. You’ll be surprised at how much value some basic automated smoke tests or automated unit tests can provide.
测试自动化是一项团队工作。这也很难,至少一开始是这样。通常需要克服一大堆“痛苦”。如果您管理开发或测试团队,请确保您以时间、培训和动力的形式提供足够的支持。如果您是没有自动化的团队中的测试人员,并且程序员过于疯狂地尝试编写生产代码而无法停下来考虑测试,那么您将面临巨大的挑战。尝试从管理层和团队成员那里获得支持的不同方式,以开始一些微小的自动化工作。
Test automation is a team effort. It’s also hard, at least at first. There’s often a big “hump of pain” to overcome. If you manage a development or testing team, make sure you’re providing enough support in the form of time, training, and motivation. If you’re a tester on a team with no automation, and the programmers are too frantic trying to write production code to stop and think about testing, you have a big challenge ahead of you. Experiment with different ways of getting support from management and from team members to start some tiny automation effort.
反馈是核心敏捷价值。敏捷的短期迭代旨在提供持续的反馈,以使团队保持在正轨上。测试人员处于独特的位置,可以帮助以自动化测试结果、探索性测试期间的发现以及对系统实际用户的观察的形式提供反馈。
Feedback is a core agile value. The short iterations of agile are designed to provide constant feedback in order to keep the team on track. Testers are in a unique position to help provide feedback in the form of automated test results, discoveries made during exploratory testing, and observations of actual users of the system.
测试人员也需要反馈。你怎么知道你有来自客户的期望行为的正确例子?您怎么知道您编写的测试用例是否正确反映了这些示例?程序员能否通过查看您捕获的示例和您创建的测试来理解要编码的内容?
Testers need feedback too. How do you know that you have the right examples of desired behavior from the customers? How do you know if the test cases you wrote reflected these examples correctly? Can the programmers understand what to code by looking at the examples you’ve captured and the tests you’ve created?
您可以学习的最有价值的技能之一是如何征求对您自己工作的反馈。询问程序员他们是否获得了足够的信息来理解需求,以及这些信息是否指导了他们的编码。询问客户是否觉得他们的质量标准得到满足。在迭代计划会议和回顾中花时间讨论这些问题并提出改进方法。
One of the most valuable skills you can learn is how to ask for feedback on your own work. Ask the programmers if they get enough information to understand requirements and whether that information guides their coding. Ask customers if they feel their quality criteria are being met. Take time in both the iteration planning meetings and retrospectives to talk about these issues and suggest ways to improve.
测试行业的一句老话是,“你不能将质量测试到产品中”。当然,敏捷开发也是如此。我们认为不遵循一些基本实践就无法交付高质量的软件。虽然我们将这些视为敏捷实践,但它们比术语“敏捷开发”存在的时间更长,而且它们只是成功软件开发的核心实践。
An old saying in the testing business is, “You can’t test quality into the product.” This is, of course, true of agile development as well. We feel you can’t deliver high-quality software without following some fundamental practices. While we think of these as agile practices, they’ve been around longer than the term “agile development,” and they’re simply core practices of successful software development.
每个开发团队都需要源代码管理和持续集成才能取得成功。如果您不确切知道,就无法进行有效测试你正在测试什么,如果你没有可以部署的代码,你根本无法测试。所有团队成员每天至少需要检查一次他们的工作。每个集成都必须通过自动构建进行验证,其中包括测试以提供有关软件状态的快速反馈。
Every development team needs source code management and continuous integration to be successful. You can’t test effectively if you don’t know exactly what you’re testing, and you can’t test at all if you have no code you can deploy. All team members need to check in their work at least once a day. Every integration must be verified by an automated build that includes tests to provide rapid feedback about the state of the software.
实施持续集成过程应该是任何软件开发团队的首要任务之一。如果您的团队没有至少一个每日验证的构建,请停止您正在做的事情并开始一个。这很重要。它不一定是完美的开始。如果您要集成一个庞大的系统,那肯定更具挑战性。不过,总的来说,这并不难。有大量优秀的工具,包括开源的和商业的,可用于此目的。
Implementing a continuous integration process should be one of the first priorities of any software development team. If your team doesn’t have at least a daily verified build, stop what you’re doing and get one started. It’s that important. It doesn’t have to be perfect to start with. If you have a huge system to integrate, it’s definitely more challenging. In general, though, it’s not that difficult. There’s a plethora of outstanding tools, both open source and commercial, available for this purpose.
如果没有您控制的测试环境,您就无法进行高效测试。您需要知道部署了什么构建,正在使用什么数据库模式,是否有其他人正在更新该模式,以及机器上正在运行哪些其他进程。
You can’t test productively without a test environment that you control. You need to know what build is deployed, what database schema is being used, whether anyone else is updating that schema, and what other processes are running on the machine.
硬件一直在变得越来越便宜,并且有更多可用于测试环境的开源软件可用。您的团队必须进行投资,以便您可以快速有效地进行自动化和手动探索性测试。如果测试环境有问题,请说出来,让团队创造性地解决这个问题。
Hardware is getting less expensive all the time, and more open source software is available that can be used for test environments. Your team must make the investment so that you can effectively conduct automated and manual exploratory tests quickly and efficiently. If there’s a problem with the test environment, speak up and let it be a problem for the team to solve creatively.
即使是优秀的软件开发团队,也会感到时间压力,忽视重构或诉诸快速修复和黑客攻击来快速解决问题。随着代码变得更加混乱和难以维护,会出现更多的错误,并且很快团队的速度就会被错误修复和试图理解代码以添加新功能所消耗。您的团队必须不断评估拖累它的技术债务数量,并努力减少和预防它。
Even good software development teams, feeling time pressure, neglect refactoring or resort to quick fixes and hacks to solve a problem quickly. As the code becomes more confusing and hard to maintain, more bugs creep in, and it doesn’t take long before the team’s velocity is consumed by bug fixes and trying to make sense out of the code in order to add new features. Your team must constantly evaluate the amount of technical debt dragging it down and work on reducing and preventing it.
人们经常说,“我们的管理层不会给我们时间把事情做好,我们没有时间重构,而且我们的期限很紧。” 然而,不难找到一个清晰的商业案例来说明不断增长的技术债务给公司带来的损失。有许多方法可以衡量代码和缺陷率,这些方法可以将技术债务转化为对底线的影响。仅仅指出你的减速速度可能就足够了。企业需要他们的软件开发团队保持始终如一的生产力。他们可能不得不缩小所需功能的范围,以便留出足够的时间进行良好的、以测试为导向的代码设计和良好实践,例如持续的小规模重构。
People often say, “Our management won’t give us time to do things right, we don’t have time to refactor, and we’re under tight deadlines.” However, it’s not hard to make a clear business case showing what growing technical debt is costing the company. There are many ways to measure code and defect rates that can translate technical debt into its impact on the bottom line. Merely pointing to your decreasing velocity may be enough. Businesses need their software development teams to remain consistently productive. They may have to reduce the scope of their desired features in order to allow enough time for good, test-guided code design and good practices such as continual small refactoring.
自动回归测试的良好覆盖率是最小化技术债务的关键。如果缺少这些,请在每次迭代中预算时间来构建自动化测试,计划“重构迭代”以升级或添加必要的工具,并编写测试和进行主要的重构工作。在每次迭代中,花时间用测试来指导代码,根据需要重构您正在接触的代码,并在缺少的地方添加自动化测试。增加您的估计以说明这项工作。从长远来看,团队将能够走得更快。
Good coverage from automated regression tests is key to minimizing technical debt. If these are lacking, budget time in each iteration to build up the automated tests, plan a “refactoring iteration” to upgrade or add necessary tools, and write tests and do major refactoring efforts. In every iteration, take the time to guide code with tests, refactor the code you’re touching as needed, and add automated tests where they’re missing. Increase your estimates to account for this work. In the long run, the team will be able to go much faster.
敏捷团队能够创造优质产品的一个原因是他们的工作规模很小。故事代表几天的工作,每个故事可能被分解成几个薄片或钢丝,并逐步构建。这允许测试每个小块,然后在将小块放在一起时进行增量测试。
One reason agile teams are able to create a quality product is that they work on a small scale. Stories represent a few days of work, and each story may be broken into several thin slices or steel threads and built step-by-step. This allows testing each small piece and then incrementally testing as the pieces are put together.
如果您的团队成员很想同时承担大量功能,请鼓励他们考虑循序渐进的方法。提出问题:“这个故事的核心商业价值是什么?这段代码最基本的路径是什么?接下来会发生什么?建议编写任务卡来编码和测试小块,获得设计的概念证明,并确认您的测试和测试自动化策略。
If your team members are tempted to take on a large chunk of functionality at once, encourage them to look at a stepwise approach. Ask questions: “What’s the central business value in this story? What’s the most basic path through this piece of code? What would come next?” Suggest writing task cards to code and test the small pieces, get a proof of concept for your design, and confirm your test and test automation strategy.
在第 8 章“支持团队的面向业务的测试”中阅读有关小块和薄片的更多信息。
Read more about small chunks and thin slices in Chapter 8, “Business-Facing Tests that Support the Team.”
刚接触敏捷的人经常问敏捷测试人员,“在所有的故事都完成并且可以测试之前,你会做什么?” 经验丰富的敏捷从业者说,“测试人员必须参与整个迭代,整个开发过程。否则是行不通的。”
People who are new to agile often ask agile testers, “What do you do until all the stories are finished and you can test?” Experienced agile practitioners say, “Testers must be involved throughout the whole iteration, the whole development process. Otherwise it doesn’t work.”
测试人员根据客户提供的示例编写测试,以帮助程序员理解故事并开始使用。测试和示例提供了参与软件生产的每个人都能理解的通用语言。随着编码的进行,测试人员和程序员密切合作,他们也都与客户密切合作。程序员向测试人员展示他们编写的功能,而测试人员向程序员展示他们发现的意外行为。随着编码的进行,测试人员编写更多测试,程序员让它们通过,测试人员会进行更多的探索性测试,以了解是否已交付正确的价值。每个敏捷迭代都由几十个持续的、快速的、增量的测试-代码-测试-代码-测试迭代组成。
Testers write tests, based on examples provided by customers, to help programmers understand the story and get started. Tests and examples provide a common language that everyone involved in producing the software understands. Testers and programmers collaborate closely as coding proceeds, and they both also collaborate closely with the customers. Programmers show testers the functionality they’ve written, and testers show programmers the unexpected behaviors they’ve found. Testers write more tests as coding proceeds, programmers make them pass, and testers do more exploratory testing to learn whether the right value has been delivered. Each agile iteration consists of dozens of constant, quick, incremental test-code-test-code-test iterations.
当这种协作和反馈周期受到干扰,并且测试与开发分离时,就会发生不好的事情。如果一个故事在编码后的迭代中进行了测试并发现了错误,程序员必须停止处理新故事,记住代码是如何在上一次迭代的故事中工作的,修复它,然后等待有人测试使固定。软件开发中的事实很少,但我们可以肯定地知道,越早发现错误,修复它们的成本就越低。
When this collaboration and feedback cycle is disturbed, and testing is separated from development, bad things happen. If a story is tested in the iteration after which it was coded and bugs are found, the programmer has to stop working on the new story, remember how the code worked for the last iteration’s story, fix it, and wait for someone to test the fix. There are few facts in software development, but we know for sure that bugs are cheaper to fix the sooner they’re found.
当编码始终由测试指导,并且测试与编码一起进行时,我们更有可能实现行为并提供客户想要的价值。测试是团队的责任。如果您的团队不同意这种观点,请让每个人思考他们对质量的关注、他们交付尽可能最好的产品的愿望,以及他们可以采取哪些步骤来确保团队实现其目标。
When coding is constantly guided by tests, and testing happens alongside coding, we’re much more likely to achieve the behavior and provide the value that the customer wanted. Testing is a team responsibility. If your team doesn’t share this view, ask everyone to think about their focus on quality, their desire to deliver the best possible product, and what steps they can take to ensure that the team achieves its goals.
在第 18 章“编码和测试”中阅读有关编码和测试的更多信息。
Read more about coding and testing in Chapter 18, “Coding and Testing.”
单一的敏捷开发实践(如持续集成)可能会有所作为,但多个敏捷实践的组合大于部分之和。测试驱动设计、集体代码所有权和持续集成一起提供快速反馈,不断改进代码设计和快速交付业务价值的能力。自动化测试很好,但是使用自动化测试来驱动开发,然后通过探索性测试来检测差距或弱点,效果要好很多。
A single agile development practice such as continuous integration can make a difference, but the combination of multiple agile practices is greater than the sum of the parts. Test-driven design, collective code ownership, and continuous integration together deliver rapid feedback, continually improving code design and the ability to deliver business value quickly. Automating tests is good, but using automated tests to drive development, followed up by exploratory testing to detect gaps or weaknesses, is many levels of magnitude better.
有些做法孤立地运作效果不佳。没有自动化测试就不可能进行重构。以迷你瀑布方式进行小版本发布并避免敏捷开发的所有好处是可能的。如果您的现场客户无权做出决定,那么她对团队的价值是有限的。
Some practices don’t work well in isolation. Refactoring is impossible without automated tests. It’s possible to do small releases in a mini-waterfall fashion and avoid all benefits of agile development. If your on-site customer isn’t empowered to make decisions, her value to the team is limited.
敏捷实践旨在相互补充。花时间了解每项实践的目的,考虑充分利用每项实践所需的条件,并就哪些对您的团队有用做出深思熟虑的决定。
Agile practices were designed to complement each other. Take time to understand the purpose of each one, consider what is needed to take full advantage of each practice, and make thoughtful decisions about what works for your team.
测试人员为敏捷团队贡献的一些最大价值是帮助客户澄清需求并确定需求的优先级,说明需求具有所需行为和用户场景的具体示例,并将这些示例转化为可执行的测试。测试人员使用业务的领域语言和开发团队的技术语言。我们是优秀的促进者和翻译者。
Some of the greatest value that testers contribute to agile teams is helping customers clarify and prioritize requirements, illustrating the requirements with concrete examples of desired behavior and user scenarios, and turning those examples into executable tests. Testers speak the domain language of the business and the technical language of the development team. We make good facilitators and translators.
永远不要妨碍程序员和客户之间的直接交流。鼓励尽可能多的直接交流。使用“三的力量”。当需求被遗漏或被误解时,客户、程序员和测试人员需要共同努力来回答问题。让客户在必要时经常在白板或虚拟等效物前交谈。如果客户分散在园区、国家或全球各地,请使用您能找到的所有工具来加强沟通和协作。电话会议、即时消息和维基不是面对面对话的理想替代品,但它们胜过发送电子邮件或根本不交谈。
Never get in the way of direct communication between programmers and customers. Do encourage as much direct communication as possible. Use the “Power of Three.” When requirements are missed or misunderstood, a customer, programmer, and tester need to work together to get questions answered. Get the customers talking in front of a whiteboard or its virtual equivalent as often as necessary. If customers are scattered around the campus, the country, or the globe, use every tool you can find to enhance communication and collaboration. Teleconferences, instant messages, and wikis aren’t an ideal replacement for face-to-face conversation, but they beat sending emails or not talking at all.
当然,这是一种概括,但我们发现测试人员倾向于从大局着眼,而且通常是从客户的角度出发。程序员通常必须专注于交付他们现在正在处理的故事,虽然他们可能会使用测试来指导他们,但他们必须专注于需求的技术实现。
This is a generalization, of course, but we’ve found that testers tend to look at the big picture, and usually from a customer point of view. Programmers usually have to focus on delivering the story they’re working on now, and while they may be using tests to guide them, they have to focus on the technical implementation of the requirements.
这种大局观是对团队的巨大贡献。测试驱动的开发如果做得好,可以交付可靠的代码,这些代码在孤立的情况下可能没有缺陷。如果该新功能导致应用程序的某些明显不相关的部分中断怎么办?必须有人考虑对更大系统的影响,并将其引起团队的注意。如果我们忽略了一些会激怒客户的小细节怎么办?新 UI 的编码可能完美无缺,但如果背景颜色使文本难以阅读,最终用户就会注意到这一点。
This big-picture viewpoint is a huge contribution to the team. Test-driven development, done well, delivers solid code that may, in isolation, be free of defects. What if that new feature causes some apparently unrelated part of the application to break? Someone has to consider the impact to the larger system and bring that to the team’s attention. What if we’ve overlooked some little detail that will irritate the customers? The new UI may be flawlessly coded, but if the background color makes the text hard to read, that’s what the end user’s going to notice.
使用敏捷测试象限作为指南,帮助您规划涵盖所有角度的测试。使用测试金字塔的想法来确保测试自动化的良好投资回报率。通过测试指导开发有助于确保您不会错过重要的事情,但这并不完美。使用探索性测试来了解有关应用程序应如何工作以及测试需要采取的方向的更多信息。使用反映真实世界的数据,使您的测试环境尽可能类似于生产环境。努力为负载测试等活动重新创建生产风格的情况。
Use the Agile Testing Quadrants as a guide to help you plan testing that will cover all the angles. Use the test pyramid idea to ensure good ROI from your test automation. Guiding development with tests helps make sure you don’t miss something big, but it’s not perfect. Use exploratory testing to learn more about how the application should work, and what direction your testing needs to take. Make your test environments as similar as possible to production, using data that reflects the real world. Be diligent about re-creating a production-style situation for activities such as load testing.
团队中的每个人都很容易狭隘地只关注手头的任务或故事。这是一次处理小块功能的缺点。帮助您的团队时不时后退一步,评估您当前的故事如何融入企业的宏伟计划。不断问自己如何才能更好地交付真正的价值。
It’s easy for everyone on the team to narrowly focus only on the task or story at hand. That’s a drawback of working on small chunks of functionality at a time. Help your team take a step back now and then to evaluate how your current stories fit into the grand scheme of the business. Keep asking yourselves how you can do a better job of delivering real value.
测试和质量是整个团队的责任,但测试人员带来了特殊的观点和独特的技能。作为一名测试人员,您对交付令客户满意的产品的热情将带您度过您和您的团队可能遇到的挫折。不要害怕成为持续改进的推动者。在与客户和开发团队合作时,让敏捷原则和价值观指导您,在每次迭代中增加价值。
Testing and quality are the responsibility of the whole team, but testers bring a special viewpoint and unique skills. As a tester, your passion for delivering a product that delights your customers will carry you through the frustrations you and your team may encounter. Don’t be afraid to be an agent for continual improvement. Let agile principles and values guide you as you work with the customer and development teams, adding value throughout each iteration.
在最后一章中,我们研究了敏捷测试成功的七个关键因素:
In this concluding chapter, we looked at seven key factors for successful agile testing:
1. Use the whole-team approach.
2 . 采用敏捷测试思维模式。
2. Adopt an agile testing mind-set.
3. Automate regression testing.
4. Provide and obtain feedback.
5. Build a foundation of core practices.
This glossary contains the authors’ definitions of terms used throughout this book.
验收测试是定义每个故事必须交付的商业价值的测试。他们可以验证功能需求或非功能需求,例如性能或可靠性。尽管它们用于帮助指导开发,但它比测试驱动开发中用于代码设计的单元级测试处于更高级别。验收测试是一个广义术语,可能包括面向业务和面向技术的测试。
Acceptance tests are tests that define the business value each story must deliver. They may verify functional requirements or nonfunctional requirements such as performance or reliability. Although they are used to help guide development, it is at a higher level than the unit-level tests used for code design in test-driven development. Acceptance test is a broad term that may include both business-facing and technology-facing tests.
API 使其他软件能够调用某些功能。API 可能包含支持其他程序发出的请求的函数、过程或类。
APIs enable other software to invoke some piece of functionality. The API may consist of functions, procedures, or classes that support requests made by other programs.
构建是将源代码转换为可部署工件的过程,可以安装该工件以运行应用程序。术语“构建”也指可部署的工件。
A build is the process of converting source code into a deployable artifact that can be installed to run the application. The term “build” also refers to the deployable artifact.
组件是整个系统的较大部分,可以单独部署。例如,在Windows平台上,动态链接库(DLL)作为组件,Java Archives(JAR文件)在Java平台上作为组件,面向服务的体系结构(SOA)使用Web Services作为组件。
A component is a larger part of the overall system that may be separately deployable. For example, on the Windows platform, dynamic linked libraries (DLLs) are used as components, Java Archives (JAR files) are components on the Java platform, and a service-oriented architecture (SOA) uses Web Services as components.
组件测试验证组件的行为。组件测试通过测试对象之间的交互来帮助组件设计。
A component test verifies a component’s behavior. Component tests help with component design by testing interactions between objects.
满意度条件,也称为满意度条件或业务满意度条件,是客户团队为定义期望的行为而做出的关键假设和决策为给定故事提供的代码。满意度条件是衡量故事结果的标准。他们在与客户就每个故事的高级验收标准进行对话时不断发展。讨论满意度条件有助于识别有风险的假设,并增加团队编写和正确估计完成故事的所有任务的信心。
Conditions of satisfaction, also called satisfaction conditions or conditions of business satisfaction, are key assumptions and decisions made by the customer team to define the desired behavior of the code delivered for a given story. Conditions of satisfaction are criteria by which the outcome of a story can be measured. They evolve during conversations with the customer about high-level acceptance criteria for each story. Discussing conditions of satisfaction helps identify risky assumptions and increases the team’s confidence in writing and correctly estimating all the tasks to complete the story.
上下文驱动测试遵循七个原则,第一个原则是任何实践的价值都取决于其上下文。每个新项目和每个新应用程序都可能需要不同的方法来处理项目。所有七种做法都可以在网站www.context-driven-testing.com/上找到。
Context-driven testing follows seven principles, the first being that the value of any practice depends on its context. Every new project and every new application may require different ways of approaching a project. All seven practices can be found on the website www.context-driven-testing.com/.
客户团队确定业务所需的功能并确定其优先级。在 Scrum 中,这些特性成为史诗或主题,进一步分解为故事并构成产品待办事项列表。客户团队包括开发团队之外的所有利益相关者,例如业务专家、主题专家和最终用户。测试人员和开发人员与客户团队密切合作,为每个故事指定所需行为的示例,并将这些示例转化为测试以指导开发。
The customer team identifies and prioritizes the features needed by the business. In Scrum, these features become epics or themes, which are further broken into stories and comprise the product backlog. Customer teams include all stakeholders outside of the development team, such as business experts, subject-matter experts, and end users. Testers and developers work closely with the customer team to specify examples of desired behavior for each story and turn those examples into tests to guide development.
客户测试验证客户可见并直接与故事或功能相关的功能片段或片段的行为。术语“面向业务的测试”和“面向客户的测试”指的是与客户测试相同类型的测试。
A customer test verifies the behavior of a slice or piece of functionality that is visible to the customer and related directly back to a story or feature. The terms “business-facing test” and “customer-facing test” refer to the same type of test as customer test.
开发团队是生产客户团队要求的软件的技术团队。参与交付软件的每个人都是开发人员,包括程序员、测试人员、数据库专家、系统管理员、技术作家、架构师、可用性专家和分析师。这个开发团队一起工作来生产软件并为企业创造价值,无论他们是同地团队还是虚拟团队。
The development team is the technical team that produces the software requested by the customer team. Everyone involved in delivering software is a developer, including programmers, testers, database experts, system administrators, technical writers, architects, usability experts, and analysts. This development team works together to produce the software and deliver value to the business, whether they are a co-located team or a virtual team.
史诗是客户描述的一项功能或特性,是产品待办事项列表中的一项。一个史诗被分解成相关的故事,然后进行大小和估计。一些团队使用术语“主题”而不是史诗。
An epic is a piece of functionality, or feature, described by the customer and is an item on the product backlog. An epic is broken up into related stories that are then sized and estimated. Some teams use the term “theme” instead of epic.
探索性测试是一种交互式测试,它将测试设计与测试执行相结合,并侧重于了解应用程序。有关探索性测试的广泛定义,请参阅第 10 章“评估产品的面向业务的测试”。
Exploratory testing is interactive testing that combines test design with test execution and focuses on learning about the application. See Chapter 10, “Business-Facing Tests that Critique the Product,” for an extensive definition of exploratory testing.
假对象用更简单的实现替换了依赖组件的功能。它模拟真实依赖组件的行为,但更易于用于测试目的。
A fake object replaces the functionality of the depended-on component with a simpler implementation. It emulates the behavior of the real depended-on component but is easier to use for testing purposes.
特性是客户描述的一项功能,是产品待办事项列表中的一项。一个功能被分解成相关的故事,然后进行大小和估计。在敏捷开发中,术语“史诗”或“主题”经常用来代替“特性”。
A feature is a piece of functionality described by the customer and is an item on the product backlog. A feature is broken up into related stories that are then sized and estimated. In agile development, the terms “epic” or “theme” are often used in place of “feature.”
功能测试验证系统在给定一组输入和/或操作的情况下的预期行为。
Functional tests verify the system’s expected behavior given a set of inputs and/or actions.
Greenfield 项目是从头开始的新应用程序开发项目,没有现有的代码库。没有任何限制,因此开发团队可以有很多选择。
Greenfield projects are new application development projects starting from scratch with no existing code base. There are no constraints, so development teams have many options open to them.
集成开发环境或 IDE 是一组支持编程和测试的工具。它通常包括编辑器、编译器或解释器调试器、重构功能和构建自动化工具。IDE 通常支持与源代码控制系统集成,并提供特定于语言的支持以帮助进行代码设计。
An Integrated Development Environment, or IDE, is a set of tools that support programming and testing. It usually includes an editor, compiler or intepreter debugger, refactoring capabilities, and build automation tools. IDEs usually enable integration with a source code control system and provide language-specific support to help with code design.
迭代是一个较短的开发周期,通常为一到四个星期,在该周期结束时可能会交付生产就绪代码。交付整个主题或史诗可能需要多次迭代,每次迭代的长度都相同。有些团队实际上每次迭代都会将代码发布到生产环境,但即使代码未发布,它也已准备好发布。
An iteration is a short development cycle, generally from one to four weeks, at the end of which production-ready code can potentially be delivered. Several iterations, each one the same length, may be needed to deliver an entire theme or epic. Some teams actually release the code to production each iteration, but even if the code isn’t released, it is ready for release.
Java 消息服务 (JMS) API 是一种消息传递标准,它使基于 Java 2 平台企业版 (J2EE) 的应用程序组件能够创建、发送、接收和读取消息。
The Java Messaging Service (JMS) API is a messaging standard that enables application components based on the Java 2 Platform, Enterprise Edition (J2EE) to create, send, receive, and read messages.
遗留系统是没有任何(或很少)自动回归测试的系统。在遗留代码中引入更改或重构它可能会有风险,因为没有测试来捕获系统行为的意外更改。
A legacy system is one that does not have any (or few) automated regression tests. Introducing changes in legacy code, or refactoring it, might be risky because there are no tests to catch unintended changes in system behavior.
多用途 Internet 邮件扩展,或 MIME,扩展 Internet 邮件的格式以启用非文本邮件、多部分邮件正文和非 US-ASCII 文本邮件和标题。
Multipurpose Internet Mail Extensions, or MIME, extend the format of Internet mail to enable non-textual messages, multipart message bodies, and non-US-ASCII textual messages and headers.
模拟对象模拟现有对象的响应。它有助于设计和测试对象之间的交互,替换真实组件,以便测试可以验证其间接输出。
A mock object simulates the responses of an existing object. It helps with designing and testing interactions between objects, replacing a real component so that a test can verify its indirect outputs.
Product Backlog 是一个 Scrum 术语,指的是产品所需所有功能的优先主列表。随着组织考虑他们可能需要的新功能,这种积压会随着时间的推移而增加。
Product Backlog is a Scrum term for the prioritized master list of all functionality desired in the product. This backlog grows over time as the organization thinks of new features they may need.
产品负责人是一个 Scrum 术语,指的是负责确定产品待办事项或故事优先级的人。他或她通常是营销人员或参与开发的关键业务专家。
Product Owner is a Scrum term for the person responsible for prioritizing the product backlog, or stories. He or she is typically someone from a marketing role or a key business expert involved with development.
质量保证或 QA 可以定义为为确保符合质量标准而采取的行动。在软件开发中,术语“QA 团队”通常用于指代进行软件测试的团队。测试团队(参见测试团队)为涉众提供与软件产品质量相关的信息。他们执行活动以了解被测系统的行为方式并验证其行为是否符合预期。在敏捷开发中,这些活动与开发活动完全集成。测试人员通常是开发团队的一部分,与其他参与软件开发的人一样。
Quality Assurance, or QA, can be defined as actions taken to ensure compliance with a quality standard. In software development, the term “QA Team” is often used to refer to the team that does software testing. Test teams (see Test Team) provide stakeholders with information related to the quality of the software product. They perform activities to learn how the system under test should behave and verify that it behaves as expected. In agile development, these activities are fully integrated with development activities. Testers are often part of the development team along with everyone else involved in developing the software.
生产代码是用于或将用于生产的系统代码,与为测试它而编写的代码不同。测试代码调用或操作生产代码以验证其行为。
Production code is the code for the system that is, or will be, used in production, as distinguished from the code that is written to test it. Test code invokes or operates on production code to verify its behavior.
重构是在不改变其功能的情况下更改代码,以使其更易于维护、更易于阅读、更易于测试或更易于扩展。
Refactoring is changing code, without changing its functionality, to make it more maintainable, easier to read, easier to test, or easier to extend.
回归测试验证被测系统的行为没有改变。回归测试通常编写为单元测试以驱动编码或验收测试以定义所需的系统行为。一旦测试通过,它们就会成为回归测试套件的一部分,以防止引入意外更改。回归测试应该是自动化的,以确保持续的反馈。
A regression test verifies that the behavior of the system under test hasn’t changed. Regression tests are usually written as unit tests to drive coding or acceptance tests to define desired system behavior. Once the tests pass, they become part of a regression test suite, to guard against unintended changes being introduced. Regression tests should be automated to ensure continual feedback.
候选发布版是可能发布到生产环境的产品版本或构建。候选发布版可能会接受进一步测试,或者通过文档或其他材料进行扩充。
A release candidate is a version or build of the product that can potentially be released to production. The release candidate may undergo further testing or be augmented with documentation or other materials.
投资回报率或 ROI 是从金融投资领域借用的术语,是衡量投资效率的指标。投资回报率可以用不同的方式计算,但它基本上是投资收益与投资成本之间的差值除以投资成本。在测试中,ROI 是从测试活动(例如自动化测试)中获得的收益,与生成和维护该测试或活动的成本进行权衡。
Return on investment, or ROI, is a term borrowed from the world of financial investments and is a measure of the efficiency of an investment. ROI can be calculated in different ways, but it’s basically the difference between the gain from an investment and the cost of that investment, divided by the cost of that investment. In testing, ROI is the benefit gained from a testing activity such as automating a test, weighed against the cost of producing and maintaining that test or activity.
SOAP 是一种通过网络交换基于 XML 的消息的协议,通常使用 HTTP/HTTPS。它构成了 Web 服务协议栈的基础层,提供了一个可以构建抽象层的基本消息传递框架。一种常见的 SOAP 消息传递模式是远程过程调用 (RPC) 模式,其中客户端网络节点向服务器节点发送请求消息,服务器立即向客户端发送响应。
SOAP is a protocol for exchanging XML-based messages over networks, normally using HTTP/HTTPS. It forms the foundation layer of the web services protocol stack, providing a basic messaging framework upon which abstract layers can be built. A common SOAP messaging pattern is the Remote Procedure Call (RPC) pattern, in which the client network node sends a request message to the server node, and the server immediately sends a response to the client.
用户故事是从用户的角度讲述的对用户或客户团队有价值的功能的简短描述。故事传统上写在索引卡上。该卡片通常包含对该功能的单行描述。例如,“作为购物者,我可以将商品放入购物车,以便稍后结账”是一个故事。卡片只能与客户团队和开发团队之间的后续对话结合使用,并通过编写和运行测试验证故事是否已实现。
A user story is a short description of functionality told from the perspective of the user that is valuable to either the user or the customer team. Stories are traditionally written on index cards. The card typically contains a one-line description of the feature. For example, “As a shopper, I can put items in my shopping cart so that I can check out with them later” is a story. Cards are only useable in combination with subsequent conversations between the customer team and the development team and some verification that the story has been implemented through writing and running tests.
故事测试定义故事交付的代码的预期行为。故事测试可能是面向业务的,指定功能需求,也可能是面向技术的,例如安全或性能测试。这些测试用于指导开发以及验证交付的代码。大多数敏捷实践者使用术语“故事测试”作为“验收测试”的同义词,尽管术语“验收测试”可能用于验证比故事更高级别的行为的测试。
A story test defines expected behavior for the code to be delivered by the story. Story tests may be business-facing, specifying the functional requirements, or technology-facing, such as security or performance tests. These tests are used to guide development as well as to verify the delivered code. Most agile practitioners use the term “story test” synonymously with “acceptance test,” although the term “acceptance test” might be used for tests that verify behavior at a higher level than one story.
故事板,也称为任务板,用于跟踪团队在迭代期间所做的工作。为每个故事编写任务卡,颜色可能与任务类型相协调。这些卡片连同某种视觉提示提供了一种简单的机制来查看迭代进度的当前状态。它可以使用列或卡片上的不同颜色贴纸代表不同的状态,例如“待办事项”、“进行中的工作”、“验证”和“完成”。故事板可以是墙上的物理板或虚拟在线板。
The story board, also called the task board, is used to track the work the team does during an iteration. Task cards, which may be color-coordinated for the type of task, are written for each story. These cards, along with a visual cue of some kind, provide an easy mechanism for seeing the current status of an iteration’s progress. It may use columns or different colored stickers on cards for different states such as “To do,” “Work in Progress,” “Verify,” and “Done.” The story board might be a physical board on a wall or a virtual online board.
任务是完成故事所需的工作。任务可能是实现故事的一小部分所需的操作,也可能是构建一些基础设施或测试包含多个故事的内容。通常它应该代表一天或更少的工作。
Tasks are pieces of work needed to finish a story. A task might be action needed to implement a small piece of a story, or it might be for building a bit of infrastructure, or testing that encompasses more than one story. Generally it should represent a day or less of work.
Ward Cunningham 首先介绍了这个比喻。当团队在不使用 TDD、持续集成和重构等良好实践的情况下生产软件时,可能会产生技术债务。与金融债务一样,技术债务会产生利息,这将使团队在以后付出更多代价。有时这笔债可能是值得的,比如利用一个突如其来的商机。不过,通常情况下,技术债务会加剧并减慢团队的速度。每次迭代中产生的商业价值越来越少,因为代码缺乏自动回归测试的安全网,或者变得难以理解和维护。
Ward Cunningham first introduced this metaphor. When a team produces software without using good practices such as TDD, continuous integration, and refactoring, it may incur technical debt. Like financial debt, technical debt accrues interest that will cost the team more at a later date. Sometimes this debt may be worthwhile, such as to take advantage of a sudden business opportunity. Usually, though, technical debt compounds and slows the team’s velocity. Less and less business value can be produced in each iteration because the code lacks a safety net of automated regression tests or has become difficult to understand and maintain.
测试替身是为了运行测试的明确目的而安装在真实组件位置的任何对象或组件。测试替身包括虚拟对象、模拟对象、测试存根和假对象。
A test double is any object or component that’s installed in place of the real component for the express purpose of running a test. Test doubles include dummy objects, mock objects, test stubs, and fake objects.
在测试驱动开发中,程序员先编写并自动化一个小的单元测试,然后再编写使测试通过的小代码段。生产代码一次只能进行一个测试。
In test-driven development, the programmer writes and automates a small unit test before writing the small piece of code that will make the test pass. The production code is made to work one test at a time.
在测试先行的开发中,测试是在相应的生产代码之前编写的,但代码不一定一次只能运行一个测试。客户或故事测试可用于测试先行开发以及单元测试。
In test-first development, tests are written in advance of the corresponding production code, but the code is not necessarily made to work one test at a time. Customer or story tests may be used in test-first development as well as unit tests.
测试存根是一个对象,它将被测系统所需的真实组件替换为特定于测试的对象,该对象将所需的间接输入提供给被测系统。这使测试能够独立于其他组件来验证逻辑。
A test stub is an object that replaces a real component needed by the system under test with a test-specific object that feeds desired indirect inputs into the system under test. This enables the test to verify logic independently of the other components.
测试团队执行的活动有助于定义并随后验证被测系统的预期行为。测试团队向利益相关者提供有关系统外部质量、可能存在的风险以及潜在的风险缓解策略的信息。在敏捷开发中,这些活动与发展活动充分结合。测试人员通常是开发团队的一部分,与其他参与软件开发的人一样。
A test team performs activities that help define and subsequently verify the desired behavior of the system under test. The test team provides information to the stakeholders about the external quality of the system, the risks that may be present, and potential risk mitigation strategies. In agile development, these activities are fully integrated with development activities. Testers are often part of the development team along with everyone else involved in developing the software.
测试人员向利益相关者提供有关正在开发的软件的信息。测试人员帮助客户定义功能和非功能需求以及质量标准,并帮助将这些转化为指导开发和验证所需行为的测试。测试人员执行与交付高质量软件相关的各种活动,例如测试自动化和探索性测试。在敏捷开发中,开发团队中的每个人都执行测试活动。自称为测试人员的团队成员与开发人员和客户团队的其他成员密切合作。
A tester provides information to stakeholders about the software being developed. A tester helps customers define functional and nonfunctional requirements and quality criteria, and helps turn these into tests that guide development and verify desired behavior. Testers perform a wide variety of activities related to delivering high-quality software, such as test automation and exploratory testing. In agile development, everyone on the development team performs testing activities. Team members who identify themselves as testers work closely with other members of both the developer and customer teams.
主题与史诗或故事片相同。它是客户描述并放置在产品待办事项列表中的一项功能,将被分解成大小和估计的故事。
A theme is the same as an epic or feature. It is a piece of functionality described by the customer and placed in the product backlog to be broken up into stories that are sized and estimated.
单元测试验证整个系统的一小部分的行为。它可能小到单个对象或方法,是一个或多个设计决策的结果。
A unit test verifies the behavior of a small part of the overall system. It may be as small as a single object or method that is a consequence of one or more design decisions.
开发团队的速度是它在每次迭代中交付的价值量,以故事点、理想天数或小时数衡量。通常,速度中只包含已完成的故事。Velocity 有助于企业规划未来的功能和版本。敏捷团队使用上一次迭代的速度来帮助确定他们在下一次迭代中可以承担的工作量。
A development team’s velocity is the amount of value it delivers in each iteration, measured in story points, ideal days, or hours. Generally, only completed stories are included in the velocity. Velocity is helpful to the business in planning for future features and releases. Agile teams use their velocity for the previous iteration to help determine the amount of work they can take on in the next iteration.
Web 服务描述语言 (WDSL) 是一种 XML 格式,用于将网络服务描述为一组端点,这些端点对包含面向文档或面向过程的信息的消息进行操作。
Web Service Description Language (WDSL) is an XML format for describing network services as a set of endpoints operating on messages containing either document-oriented or procedure-oriented information.
敏捷联盟。“敏捷宣言背后的原则”,www.agilemanifesto.org/principles.html,2001 年。
Agile Alliance. “Principles Behind the Agile Manifesto,” www.agilemanifesto.org/principles.html, 2001.
Alles、Micah、David Crosby、Carl Erickson、Brian Harleton、Michael Marsiglia、Greg Pattison 和 Curt Stienstra。“Presenter First:为测试驱动开发组织复杂的 GUI 应用程序”,Agile 2006,明尼苏达州明尼阿波利斯,2006 年 7 月。
Alles, Micah, David Crosby, Carl Erickson, Brian Harleton, Michael Marsiglia, Greg Pattison, and Curt Stienstra. “Presenter First: Organizing Complex GUI Applications for Test-Driven Development,” Agile 2006, Minneapolis, MN, July 2006.
安布勒,斯科特。敏捷数据库技术:敏捷软件开发人员的有效策略,Wiley,2003 年。
Ambler, Scott. Agile Database Techniques: Effective Strategies for the Agile Software Developer, Wiley, 2003.
阿斯特尔斯,大卫。测试驱动开发:实用指南,Prentice Hall,2003 年。
Astels, David. Test-Driven Development: A Practical Guide, Prentice Hall, 2003.
巴赫,詹姆斯。“解释性测试”,www.satisfice.com/articles/ et-article.pdf,2003 年。
Bach, James. “Exploratory Testing Explained,” www.satisfice.com/articles/et-article.pdf, 2003.
巴赫,乔纳森。“基于会话的测试管理”,软件测试和质量工程杂志,2000 年 11 月,www.satisfice.com/ articles/sbtm.pdf 。
Bach, Jonathan. “Session-Based Test Management,” Software Testing and Quality Engineering Magazine, November, 2000, www.satisfice.com/articles/sbtm.pdf.
贝克,肯特。极限编程解释:拥抱变化,Addison-Wesley,2000 年。
Beck, Kent. Extreme Programming Explained: Embrace Change, Addison-Wesley, 2000.
贝克,肯特和安德烈斯,辛西娅。极限编程解释:拥抱变化。第 2 版,Addison-Wesley,2004 年。
Beck, Kent, and Andres, Cynthia. Extreme Programming Explained: Embrace Change. 2nd Edition, Addison-Wesley, 2004.
Berczuk、Stephen 和 Brad Appleton。软件配置管理模式:有效的团队合作,实用集成,Addison-Wesley,2003 年。
Berczuk, Stephen and Brad Appleton. Software Configuration Management Patterns: Effective Teamwork, Practical Integration, Addison-Wesley, 2003.
博尔顿,迈克尔。“不使用地图进行测试”,Better Software,2005 年 1 月,www.developsense.com /articles/Testing%20Without%20A%20Map.pdf 。
Bolton, Michael. “Testing Without a Map,” Better Software, January 2005, www.developsense.com/articles/Testing%20Without%20A%20Map.pdf.
Bos、Erik 和 Christ Vriens。“敏捷 CMM”,摘自极限编程和敏捷方法 – XP/Agile Universe 2004,第四届极限编程和敏捷方法会议,加拿大卡尔加里,2004 年 8 月 15 日至 18 日,会议记录,编辑。Carmen Zannier、Hakan Erdogmus、Lowell Lindstrom,第 129–138 页,Springer,2004 年。
Bos, Erik and Christ Vriens. “An Agile CMM,” in Extreme Programming and Agile Methods–XP/Agile Universe 2004, 4th Conference on Extreme Programming and Agile Methods, Calgary, Canada, August 15–18, 2004, Proceedings, ed. Carmen Zannier, Hakan Erdogmus, Lowell Lindstrom, pp. 129–138, Springer, 2004.
Boutelle,乔纳森。“敏捷开发的可用性测试”,www.jonathanboutelle.com/ mt/archives/2005/08/usability_testi_1.html,2005 年。
Boutelle, Jonathan. “Usability Testing for Agile Development,” www.jonathanboutelle.com/mt/archives/2005/08/usability_testi_1.html, 2005.
布朗,泰特斯。“(缺乏)测试死亡螺旋”,http://ivory.idyll.org/blog/mar-08/software-quality-death-spiral.html,2008年。
Brown, Titus. “The (Lack of) Testing Death Spiral,” http://ivory.idyll.org/blog/mar-08/software-quality-death-spiral.html, 2008.
布瓦尔达,汉斯。“肥皂剧测试”,Better Software Magazine,2004 年 2 月,www.logigear.com/ resources/articles_lg/soap_opera_testing.asp 。
Buwalda, Hans. “Soap Opera Testing,” Better Software Magazine, February 2004, www.logigear.com/resources/articles_lg/soap_opera_testing.asp.
克拉克,迈克。实用项目自动化:如何构建、部署和监控 Java 应用程序,实用程序员,2004 年。
Clark, Mike. Pragmatic Project Automation: How to Build, Deploy and Monitor Java Apps, The Pragmatic Programmers, 2004.
科恩,迈克。应用于敏捷软件开发的用户故事,Addison-Wesley,2004 年。
Cohn, Mike. User Stories Applied for Agile Software Development, Addison-Wesley, 2004.
科恩,迈克。敏捷估算和规划,Prentice Hall,2005 年。
Cohn, Mike. Agile Estimating and Planning, Prentice Hall, 2005.
Crispin、Lisa 和 Tip House。测试极限编程,Addison-Wesley 2002。
Crispin, Lisa and Tip House. Testing Extreme Programming, Addison-Wesley 2002.
克里斯平,丽莎。文章“雇用敏捷测试人员”、“Web 测试工具的敏捷工具选择策略”、“提高软件质量:测试驱动开发如何影响软件质量”,http: //lisa.crispin.home.att.net。
Crispin, Lisa. Articles “Hiring an Agile Tester,” “An Agile Tool Selection Strategy for Web Testing Tools,” “Driving Software Quality: How Test-Driven Development Impacts Software Quality,” http://lisa.crispin.home.att.net.
DeMarco、Tom 和 Timothy Lister。管理软件项目的风险,Dorset House,2003 年。
DeMarco, Tom and Timothy Lister. Managing Risk on Software Projects, Dorset House, 2003.
德比,以斯帖和拉森,戴安娜。敏捷回顾:让优秀的团队更伟大,Pragmatic Bookshelf,2006 年。
Derby, Esther and Larsen, Diana. Agile Retrospectives: Making Good Teams Great, Pragmatic Bookshelf, 2006.
德比,以斯帖和罗斯曼,约翰娜。闭门造车:伟大管理的秘密,实用书架,2006 年。
Derby, Esther and Rothman, Johanna. Behind Closed Doors: Secrets of Great Management, Pragmatic Bookshelf, 2006.
德索萨,肯。“穿着开发人员外衣的测试员”博客,http://kendesouza.blogspot.com。
De Souza, Ken. “A tester in developer’s clothes” blog, http://kendesouza.blogspot.com.
Dustin、Elfriede、Chris Wysopal、Lucas Nelson 和 Dino Dia Zovi。软件安全测试的艺术:识别软件安全缺陷,赛门铁克出版社,2006 年。
Dustin, Elfriede, Chris Wysopal, Lucas Nelson, and Dino Dia Zovi. The Art of Software Security Testing: Identifying Software Security Flaws, Symantec Press, 2006.
达斯汀,埃尔弗里德。“Teamwork Tacks the Quality Goal”,软件测试与性能,第 2 卷,第 200 期,2005 年 3 月。
Dustin, Elfriede. “Teamwork Tackles the Quality Goal,” Software Test & Performance, Volume 2, Issue 200, March 2005.
Duvall、Paul、Steve Matyas 和 Andrew Glover。持续集成:提高软件质量和降低风险,Addison-Wesley,2007 年。
Duvall, Paul, Steve Matyas, and Andrew Glover. Continuous Integration: Improving Software Quality and Reducing Risk, Addison-Wesley, 2007.
埃克斯坦,尤塔。大型敏捷软件开发:深入研究,Dorset House,2004 年。
Eckstein, Jutta. Agile Software Development in the Large: Diving Into the Deep, Dorset House, 2004.
埃文斯,埃里克。领域驱动设计:应对软件核心的复杂性,Addison-Wesley,2003 年。
Evans, Eric. Domain-Driven Design: Tackling Complexity in the Heart of Software, Addison-Wesley, 2003.
羽毛,迈克尔。有效地使用遗留代码,Prentice Hall,2004 年。
Feathers, Michael. Working Effectively with Legacy Code, Prentice Hall, 2004.
弗里曼、史蒂夫和纳特普赖斯。“模拟对象”,www.mockobjects.com。
Freeman, Steve and Nat Pryce. “Mock Objects,” www.mockobjects.com.
福勒,马丁。“持续集成”,http://martinfowler.com/articles/continuousIntegration.html,2006年。
Fowler, Martin. “Continuous Integration,” http://martinfowler.com/articles/continuousIntegration.html, 2006.
福勒,马丁。“StranglerApplication”,www.martinfowler.com/bliki /StranglerApplication.html,2004 年。
Fowler, Martin. “StranglerApplication,” www.martinfowler.com/bliki/StranglerApplication.html, 2004.
Fowler, Martin,“TechnicalDebt”, http: //martinfowler.com/bliki/TechnicalDebt.html,2003年。
Fowler, Martin, “TechnicalDebt,” http://martinfowler.com/bliki/TechnicalDebt.html, 2003.
Gårtner, Markus,个人通讯,2008 年。有关更多有趣的信息,请访问他的博客:http: //blog.shino.de。
Gårtner, Markus, personal communication, 2008. For more interesting information, visit his blog at: http://blog.shino.de.
盖伦,罗伯特。Software Endgames: Eliminating Defects, Controlling Change, and the Countdown to On-Time Delivery,Dorset House,2005 年。
Galen, Robert. Software Endgames: Eliminating Defects, Controlling Change, and the Countdown to On-Time Delivery, Dorset House, 2005.
Ghiorghiu,格里格。“性能与负载与压力测试”,http://agiletesting.blogspot.com/2005/02/performance-vs-load-vs-stress-testing.html,2005年。
Ghiorghiu, Grig. “Performance vs. load vs. stress testing,” http://agiletesting.blogspot.com/2005/02/performance-vs-load-vs-stress-testing.html, 2005.
Ghirghiu,格里格。“敏捷测试”博客,http://agiletesting.blogspot.com。
Ghirghiu, Grig. “Agile Testing” blog, http://agiletesting.blogspot.com.
夏甲,乔恩。软件测试论文,www.swtesting.com/hagar_papers_index.html。
Hagar, Jon. Software Testing Papers, www.swtesting.com/hagar_papers_index.html.
亨德里克森,伊丽莎白。“测试开发人员,开发测试人员”,http://testobsessed.com/2007/01/17/tester-developers-developer-testers/,2007年。
Hendrickson, Elisabeth. “Tester Developers, Developer Testers,” http://testobsessed.com/2007/01/17/tester-developers-developer-testers/, 2007.
亨德里克森,伊丽莎白。“测试启发式备忘单”,http://testobsessed.com/wordpress/wp-content/uploads/2007/02/testheuristicscheatsheetv1.pdf,2007年。
Hendrickson, Elisabeth. “Test Heuristics Cheat Sheet,” http://testobsessed.com/wordpress/wp-content/uploads/2007/02/testheuristicscheatsheetv1.pdf, 2007.
亨德里克森,伊丽莎白。“敏捷友好型测试自动化工具/框架”,http://testobsessed.com/2008/04/29/agile-friendly-test-automation-toolsframeworks,2008年。
Hendrickson, Elisabeth. “Agile-Friendly Test Automation Tools/Frameworks,” http://testobsessed.com/2008/04/29/agile-friendly-test-automation-toolsframeworks, 2008.
海史密斯,吉姆。敏捷项目管理:创造创新产品,Addison-Wesley,2004 年。
Highsmith, Jim. Agile Project Management: Creating Innovative Products, Addison-Wesley, 2004.
亨特、安德鲁和大卫·托马斯。务实的程序员:从熟练工到大师,Addison-Wesley,1999 年。
Hunt, Andrew and David Thomas. The Pragmatic Programmer: From Journeyman to Master, Addison-Wesley, 1999.
Kaner、Cem、James Bach 和 Bret Pettichord。Lessons Learned in Software Testing,Wiley,2001 年。
Kaner, Cem, James Bach, and Bret Pettichord. Lessons Learned in Software Testing, Wiley, 2001.
克思,诺曼。项目回顾:团队审查手册,Dorset House,2001 年。
Kerth, Norman. Project Retrospectives: A Handbook for Team Reviews, Dorset House, 2001.
克尼伯格,亨里克。“如何赶上测试自动化”,http://blog.crisp.se/henrikkniberg/2008/01/03/1199386980000.html,2008年。
Kniberg, Henrik. “How to Catch Up on Test Automation,” http://blog.crisp.se/henrikkniberg/2008/01/03/1199386980000.html, 2008.
克尼伯格,亨里克。Scrum 和 XP from the Trenches,Lulu.com,2007 年。
Kniberg, Henrik. Scrum and XP from the Trenches, Lulu.com, 2007.
Koenig、Dierk、Andrew Glover、Paul King、Guillaume Laforge 和 Jon Skeet。Groovy in Action,曼宁出版社,2007 年。
Koenig, Dierk, Andrew Glover, Paul King, Guillaume Laforge, and Jon Skeet. Groovy in Action, Manning Publications, 2007.
Kohl, Jonathan。“人与机器”,Better Software杂志,2007 年 12 月。
Kohl, Jonathan.“Man and Machine,” Better Software magazine, December 2007.
科尔,乔纳森。博客和文章,www.kohl.ca/。
Kohl, Jonathan. Blog and articles, www.kohl.ca/.
卢维翁,克里斯托夫。博客,www.runningagile.com。
Louvion, Christophe. Blog, www.runningagile.com.
曼斯、玛丽·林恩和琳达·瑞辛。无所畏惧的变革:引入新想法的模式,Addison-Wesley,2004 年。
Manns, Mary Lynn and Linda Rising. Fearless Change: Patterns for Introducing New Ideas, Addison-Wesley, 2004.
马里克,布赖恩。Everyday Scripting with Ruby: For Teams, Testers and You , Pragmatic Bookshelf, 2007.
Marick, Brian. Everyday Scripting with Ruby: For Teams, Testers and You, Pragmatic Bookshelf, 2007.
Marick, Brian,“我的敏捷测试项目”,www.exampler.com/ old-blog/2003/08/21/,2003 年。
Marick, Brian, “My Agile Testing Project,” www.exampler.com/old-blog/2003/08/21/, 2003.
马里克,布赖恩。“面向业务的 TDD 的替代方案”,www.exampler.com/blog /category/aa-ftt,2008 年。
Marick, Brian. “An Alternative to Business-Facing TDD,” www.exampler.com/blog/category/aa-ftt, 2008.
马里克,布赖恩。关于敏捷测试的博客和文章,http://exampler.com。
Marick, Brian. Blog and articles on agile testing, http://exampler.com.
马尔卡诺,安东尼。博客,www.testingreflections.com。
Marcano, Antony. Blog, www.testingreflections.com.
梅萨罗斯,杰拉德。XUnit 测试模式:重构测试代码,Addison-Wesley,2007 年。
Meszaros, Gerard. XUnit Test Patterns: Refactoring Test Code, Addison-Wesley, 2007.
Meszaros、Gerard 和 Janice Aston。“将可用性测试添加到敏捷项目”,Agile 2006,明尼苏达州明尼阿波利斯,2006 年,http://papers.gerardmeszaros.com/AgileUsabilityPaper.pdf。
Meszaros, Gerard and Janice Aston. “Adding Usability Testing to an Agile Project,” Agile 2006, Minneapolis, MN, 2006, http://papers.gerardmeszaros.com/AgileUsabilityPaper.pdf.
Meszaros、Gerard、Ralph Bohnet 和 Jennitta Andrea。“使用记录和回放进行敏捷回归测试”,XP/Agile Universe 2003,路易斯安那州新奥尔良,2003 年,http://agileregressiontestpaper.gerardmeszaros.com。
Meszaros, Gerard, Ralph Bohnet, and Jennitta Andrea. “Agile Regression Testing Using Record & Playback,” XP/Agile Universe 2003, New Orleans, LA, 2003, http://agileregressiontestpaper.gerardmeszaros.com.
梅萨罗斯,杰拉德。“使用故事类型拆分臃肿的 XP 故事”,http://storyotypespaper.gerardmeszaros.com。
Meszaros, Gerard. “Using Storyotypes to Split Bloated XP Stories,” http://storyotypespaper.gerardmeszaros.com.
马格里奇、里克和沃德坎宁安。适合开发软件:集成测试框架,Prentice Hall,2005 年。
Mugridge, Rick and Ward Cunningham. Fit for Developing Software: Framework for Integrated Tests, Prentice Hall, 2005.
纽柯克、詹姆斯和阿列克谢·沃龙佐夫。Microsoft .NET 中的测试驱动开发,Microsoft Professional,2004。
Newkirk, James and Alexei Vorontsov. Test-Driven Development in Microsoft .NET, Microsoft Professional, 2004.
尼尔森,雅各布。“可用性会议的时间预算”,www.useit.com/alertbox /usability_sessions.html,2005 年。
Nielsen, Jakob. “Time Budgets for Usability Sessions,” www.useit.com/alertbox/usability_sessions.html, 2005.
北方,丹。“BDD 简介”,http://dannorth.net/introducing-bdd,2006年。
North, Dan. “Introducing BDD,” http://dannorth.net/introducing-bdd, 2006.
Patterson、Kerry、Joseph Gernny、Ben McMillan、Al Switzler 和 Stephen R. Covey。关键对话:利害攸关时的谈话工具,McGraw-Hill,2002 年。
Patterson, Kerry, Joseph Gernny, Ben McMillan, Al Switzler and Stephen R. Covey. Crucial Conversations: Tools for Talking when the Stakes are High, McGraw-Hill, 2002.
巴顿,杰夫。“编码前测试软件”,StickyMinds.com,2006 年 8 月,www.stickyminds.com/ sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=11104 。
Patton, Jeff. “Test Software Before You Code,” StickyMinds.com, August 2006, www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=11104.
巴顿,杰夫。“整体敏捷产品设计和开发”,www.agileproductdesign.com/blog /agile_product_development.html,2006 年。
Patton, Jeff. “Holistic Agile Product Design and Development,” www.agileproductdesign.com/blog/agile_product_development.html, 2006.
波尔斯,安迪。“完美的客户”,www.pols.co.uk/archives /category/testing,2008 年。
Pols, Andy. “The Perfect Customer,” www.pols.co.uk/archives/category/testing, 2008.
佩蒂索德,布雷特。“自制测试自动化”,www.io.com/~ wazmo/papers/homebrew_test_automation_200409.pdf,2004 年。
Pettichord, Bret. “Homebrew Test Automation,” www.io.com/~wazmo/papers/homebrew_test_automation_200409.pdf, 2004.
佩蒂索德,布雷特。“测试自动化成功的七个步骤”,www.io.com/~wazmo /papers/seven_steps.html,2001 年。
Pettichord, Bret. “Seven Steps to Test Automation Success,” www.io.com/~wazmo/papers/seven_steps.html, 2001.
Poppendieck、Mary 和 Tom Poppendieck。实施精益软件开发:从概念到现金,Addison-Wesley,2006 年。
Poppendieck, Mary and Tom Poppendieck. Implementing Lean Software Development: From Concept to Cash, Addison-Wesley, 2006.
Poppendieck、Mary 和 Tom Poppendieck。精益软件开发:敏捷工具包,Addison-Wesley,2003 年。
Poppendieck, Mary and Tom Poppendieck. Lean Software Development: An Agile Toolkit, Addison-Wesley, 2003.
Rainsberger,JB JUnit Recipes:程序员测试的实用方法,Manning Publications,2004 年。
Rainsberger, J. B. JUnit Recipes: Practical Methods for Programmer Testing, Manning Publications, 2004.
拉斯穆森,乔纳森。“将 XP 引入全新项目:经验教训”,IEEE Software,2003 年,http://rasmusson.files.wordpress.com/2008/01/s3021.pdf。
Rasmusson, Jonathan. “Introducing XP into Greenfield Projects: Lessons Learned,” IEEE Software, 2003, http://rasmusson.files.wordpress.com/2008/01/s3021.pdf.
罗宾斯、斯蒂芬和蒂姆·贾奇。组织行为学要点,第 9 版,Prentice Hall,2007 年。
Robbins, Stephen and Tim Judge. Essentials of Organizational Behavior, 9th Edition, Prentice Hall, 2007.
施瓦伯,肯。使用 Scrum 进行敏捷项目管理,Microsoft Press,2004 年。
Schwaber, Ken. Agile Project Management with Scrum, Microsoft Press, 2004.
Shore、James 和 Shane Warden。敏捷开发的艺术,O'Reilly Media,2007 年。
Shore, James and Shane Warden. The Art of Agile Development, O’Reilly Media, 2007.
索尼,穆克什。“缺陷预防:降低成本和提高质量”,iSixSigma,http://software.isixsigma.com/library/content/c060719b.asp。
Soni, Mukesh. “Defect Prevention: Reducing Costs and Enhancing Quality,” iSixSigma, http://software.isixsigma.com/library/content/c060719b.asp.
萨姆雷尔,梅根。“'Shout-Out' 鞋盒 – 提升团队士气”,http://megansumrell.wordpress.com/2007/08/27/shout-out-shoebox-boosting-team-morale,2007年。
Sumrell, Megan. “’Shout-Out’ Shoebox – Boosting Team Morale,” http://megansumrell.wordpress.com/2007/08/27/shout-out-shoebox-boosting-team-morale, 2007.
Sutherland、Jeff、Carsten Ruseng Jakobsen 和 Kent Johnson。“Scrum 和 CMMI 第 5 级:代码勇士的魔药”,Agile 2007,华盛顿特区,2007 年,http://jeffsutherland.com/scrum/Sutherland-ScrumCMMI6pages.pdf。
Sutherland, Jeff, Carsten Ruseng Jakobsen, and Kent Johnson. “Scrum and CMMI Level 5: The Magic Potion for Code Warriors,” Agile 2007, Washington, DC, 2007, http://jeffsutherland.com/scrum/Sutherland-ScrumCMMI6pages.pdf.
塔巴卡,让。协作说明:软件项目负责人的促进技巧,Addison-Wesley,2006 年。
Tabaka, Jean. Collaboration Explained: Facilitation Skills for Software Project Leaders, Addison-Wesley, 2006.
托马斯,迈克。“扼杀遗留代码”,Better Software杂志,2005 年 10 月,http://samoht.com/wiki_downloads/StranglingLegacyCodeArticle.pdf。
Thomas, Mike. “Strangling Legacy Code,” Better Software magazine, October 2005, http://samoht.com/wiki_downloads/StranglingLegacyCodeArticle.pdf.
托尔夫森,迈克。“客户拥护者的崛起”,STAREAST,2008 年 5 月 7-9 日。
Tholfsen, Mike. “The Rise of the Customer Champions,” STAREAST, May 7–9, 2008.
沃里斯,约翰。用于外部原型制作和测试的 ADEPT AS400 显示器,www.AdeptTesting.org。
Voris, John. ADEPT AS400 Displays for External Prototyping and Testing, www.AdeptTesting.org.
醒醒,比尔。“XP 雷达图”,http://xp123.com/xplor/xp0012b/index.shtml,2001年。
Wake, Bill. “XP Radar Chart,” http://xp123.com/xplor/xp0012b/index.shtml, 2001.
Wilson-Welsh、Patrick 和 Lisa Crispin,“翻转三角形:通向最佳、成本最低的自动化测试的路径”,Agile 2008,2008,http: //patrickwilsonwelsh.com/wp-content/uploads/2008/08/flipping-三角形.pdf。
Wilson-Welsh, Patrick and Lisa Crispin, “Flipping the Triangle: Paths to Best, Least-Cost Automated Testing”, Agile 2008, 2008, http://patrickwilsonwelsh.com/wp-content/uploads/2008/08/flipping-the-triangle.pdf.
弗里恩斯,基督。“使用 XP@Scrum 认证 CMM 2 级和 ISO9001”,载于 ADC 2003:敏捷开发会议记录,2003 年 6 月 25-28 日,美国犹他州盐湖城,120-124,IEEE,2003 年。
Vriens, Christ. “Certifying for CMM Level 2 and ISO9001 with XP@Scrum,” in ADC 2003: Proceedings of the Agile Development Conference, 25–28 June 2003, Salt Lake City, UT, USA, 120–124, IEEE, 2003.
Abbot Java GUI 测试框架,http://abbot.sourceforge.net/doc/overview.shtml。
Abbot Java GUI Test Framework, http://abbot.sourceforge.net/doc/overview.shtml.
阿兹克,戈伊科。DbFit:测试驱动的数据库开发,http://gojko.net/fitnesse/dbfit/。
Adzik, Gojko. DbFit: Test-driven Database Development, http://gojko.net/fitnesse/dbfit/.
Fauto,丹尼。“测试工具列表”,http ://testingfaqs.org,2008 年。
Faught, Danny. “Test Tools List,” http://testingfaqs.org, 2008.
Canoo WebTest,用于自动测试 Web 应用程序的开源工具,http://webtest.canoo.com。
Canoo WebTest, Open Source Tool for Automated Testing of Web Applications, http://webtest.canoo.com.
easyb,Java 平台的行为驱动开发框架,www.easyb.org/。
easyb, Behavior Driven Development Framework for the Java Platform, www.easyb.org/.
Fit,集成测试框架,http://fit.c2.com。
Fit, Framework for Integrated Test, http://fit.c2.com.
JUnit,测试驱动开发资源,www.junit.org。
JUnit, Resources for Test-Driven Development, www.junit.org.
JUnitPerf,用于性能和可扩展性测试的 JUnit 测试装饰器,http://clarkware.com/software/JUnitPerf.html。
JUnitPerf, JUnit Test Decorators for Performance and Scalability Testing, http://clarkware.com/software/JUnitPerf.html.
FitNesse,完全集成的独立 Wiki 和验收测试框架,www.fitnesse.org。
FitNesse, Fully Integrated Standalone Wiki and Acceptance Testing Framework, www.fitnesse.org.
Hower, Rick,软件 QA 和测试工具信息,www.softwareqatest.com/qattls1.html。
Hower, Rick, Software QA and Testing Tools Info, www.softwareqatest.com/qattls1.html.
NUnit,.NET 语言的单元测试框架,http://nunit.org/index.php。
NUnit, Unit-testing Framework for .NET Languages, http://nunit.org/index.php.
开源软件测试工具、新闻和讨论。www.opensourcetesting.org/。
Open Source Software Testing Tools, News and Discussion. www.opensourcetesting.org/.
RpgUnit,RPG 回归测试框架,www.RPGunit.org。
RpgUnit, RPG Regression Testing Framework, www.RPGunit.org.
Selenium,Web 应用程序测试系统,http://seleniumhq.org/。
Selenium, Web Application Testing System, http://seleniumhq.org/.
soapUI,Web 服务测试工具,www.soapui.org。
soapUI, Web Services Testing Tool, www.soapui.org.
源配置管理,http://better-scm.berlios.de。
Source Configuration Management, http://better-scm.berlios.de.
Subversion,开源版本控制系统,http://subversion.tigris.org/。
Subversion, Open Source Version Control System, http://subversion.tigris.org/.
单元测试框架。http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks。
Unit Testing Frameworks. http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks.
Watir,Ruby 中的 Web 应用程序测试,www.watir.com。
Watir, Web Application Testing in Ruby, www.watir.com.
Abbot GUI 测试工具,127
Abbot GUI test tool, 127
验收测试。另请参阅 面向业务的测试
Acceptance tests. See also Business-facing tests
定义,501
definition, 501
远程数据监控系统示例,245
Remote Data Monitoring system example, 245
UAT(用户验收测试)相比,130
UAT (user acceptance testing) compared with, 130
临时测试,198
Ad hoc testing, 198
适应能力、技能和,39–40
Adaptability, skills and, 39–40
ADEPT(用于外部原型制作和测试的 AS400 显示器),117–118
ADEPT (AS400 Displays for External Prototyping and Testing), 117–118
提前澄清
Advance clarity
客户用同一种声音说话,373–374
customers speaking with one voice, 373–374
确定故事大小,375–376
determining story size, 375–376
收集有关需求的所有观点,374-375
gathering all viewpoints regarding requirements, 374–375
提前准备
Advance preparation
缺点,373
downside of, 373
需要多少,372–373
how much needed, 372–373
敏捷开发
Agile development
敏捷宣言和,3-4
Agile manifesto and, 3–4
的障碍。请参阅 采用敏捷开发的障碍
barriers to. See Barriers to adopting agile development
团队定位,6
team orientation of, 6
Agile Estimating and Planning (Cohn), 331, 332
敏捷宣言
Agile manifesto
以人为本,30
people focus, 30
声明,4
statement of, 4
中的价值陈述,21
value statements in, 21
敏捷原则。请参阅 适用于敏捷测试人员的原则
Agile principles. See Principles, for agile testers
敏捷测试人员。另见 测试员
Agile testers. See also Testers
敏捷测试心态,482–483
agile testing mind-set, 482–483
定义,4
definition, 4
给予所有团队成员同等的权重,31
giving all team members equal weight, 31
招聘,67–69 岁
hiring, 67–69
他们是什么,19-20
what they are, 19–20
敏捷测试
Agile testing
定义,6
definition, 6
作为心态,20-21
as mind-set, 20–21
我们的意思,4-7
what we mean, 4–7
敏捷价值观,3–4
Agile values, 3–4
Alcea 的 FIT IssueTrack,84
Alcea’s FIT IssueTrack, 84
阿尔法测试,466–467
Alpha tests, 466–467
蚂蚁,284
ant, 284
作为构建工具,126
as build tool, 126
continual builds and, 175, 291
蚁丘临, 126
AnthillPro, 126
ANTS Profiler Pro, 234
ANTS Profiler Pro, 234
阿帕奇 JMeter。参见 JMeter
Apache JMeter. See JMeter
API 层功能测试工具,168–170
API-layer functional test tools, 168–170
Fit 和 FitNesse,168–170
Fit and FitNesse, 168–170
概述, 168
overview of, 168
测试网络服务,170
testing web Services, 170
API测试
API testing
自动化, 282
automating, 282
概述,205-206
overview of, 205–206
API(应用程序编程接口),501
APIs (application programming interfaces), 501
阿普尔顿,布拉德,124 岁
Appleton, Brad, 124
被测应用程序 (AUT),246
Application under test (AUT), 246
与外部应用程序的集成测试,459
integration testing with external applications, 459
远程数据监控系统示例,242–243
Remote Data Monitoring system example, 242–243
建筑学
Architecture
增量测试方法,114
incremental approach to testing, 114
分层,116
layered, 116
象限 1 测试和,99
Quadrant 1 tests and, 99
可测试, 30 , 115 , 182 , 184 , 267
testable, 30, 115, 182, 184, 267
用于外部原型制作和测试的 AS400 显示器 (ADEPT),117–118
AS400 Displays for External Prototyping and Testing (ADEPT), 117–118
假设,隐藏
Assumptions, hidden
敏捷测试人员对25 的回应
agile testers response to, 25
未能检测到,32
failure to detect, 32
发现的问题,136
questions that uncover, 136
最坏情况和334
worst-case scenarios and, 334
态度
Attitude
敏捷测试心态,482–483
agile testing mind-set, 482–483
采用敏捷开发的障碍,48
barriers to adopting agile development, 48
与技能,20
vs. skills, 20
审计,遵守审计要求,89-90
Audits, compliance with audit requirements, 89–90
AUT (被测应用程序)、143、225、246、317
AUT (application under test), 143, 225, 246, 317
授权、安全测试和,224
Authorization, security testing and, 224
自动化回归测试
Automated regression testing
关键成功因素,484
key success factors, 484
发布候选人和458
release candidates and, 458
作为安全网,261–262
as a safety net, 261–262
自动化测试列表,测试计划备选方案,353–354
Automated test lists, test plan alternatives, 353–354
自动化
Automation
代码通量和,269
code flux and, 269
部署,232
of deployment, 232
推动发展,262–263
driving development with, 262–263
探索性测试,201
of exploratory testing, 201
害怕,269–270
fear of, 269–270
来自262的反馈
feedback from, 262
解放人们从事其他工作,259–261
freeing people for other work, 259–261
功能测试结构,245–247
of functional test structure, 245–247
自制测试,175
home-brewed test, 175
所需投资,267–268
investment required, 267–268
学习曲线,266–267
learning curve, 266–267
遗留代码和,269
legacy code and, 269
可维护性和227–228
maintainability and, 227–228
手动测试与258–259
manual testing vs., 258–259
障碍,264–265
obstacles to, 264–265
旧习惯,270
old habits and, 270
概述, 255
overview of, 255
程序员的态度,265–266
programmers’ attitude regarding, 265–266
原因,257-258
reasons for, 257–258
应对变化,29
responding to change and, 29
投资回报率和264
ROI and, 264
任务卡和394–395
task cards and, 394–395
可测试性149–150
testability and, 149–150
测试作为文档,263–264
tests as documentation, 263–264
敏捷编码实践和303–304
agile coding practices and, 303–304
一次使用一个工具,312–313
applying one tool at a time, 312–313
数据生成工具,304–305
data generation tools, 304–305
数据库访问和,306-310
database access and, 306–310
设计和维护以及292–294
design and maintenance and, 292–294
发展中,288–289
developing, 288–289
确定工具要求,311–312
identifying tool requirements, 311–312
实施,316–319
implementing, 316–319
迭代方法,299–300
iterative approach, 299–300
保持简单,298–299
keep it simple, 298–299
边做边学,303
learning by doing, 303
管理自动化测试,319
managing automated tests, 319
多层方法,290-292
multi-layered approach to, 290–292
组织测试结果,322–324
organizing test results, 322–324
组织测试,319–322
organizing tests, 319–322
概述, 273
overview of, 273
原则,298
principles, 298
record/playback tools and, 294, 296–297
从最痛的区域开始,289-290
starting with area of greatest pain, 289–290
花时间做对,301-303
taking time to do it right, 301–303
测试自动化金字塔,276–279
test automation pyramid, 276–279
测试类别,274–276
test categories, 274–276
工具选择,294-298,313-316
tool selection, 294–298, 313–316
理解测试的目的,310–311
understanding purpose of tests and, 310–311
什么可以自动化,279–285
what can be automated, 279–285
什么可能难以自动化,287–288
what might be difficult to automate, 287–288
什么不应该自动化,285–287
what should not be automated, 285–287
整个团队方法,300–301
whole team approach, 300–301
自动化工具,164–177
Automation tools, 164–177
API 层功能测试工具,168–170
API-layer functional test tools, 168–170
建立和,126
builds and, 126
GUI 测试工具,170–176
GUI test tools, 170–176
概述,164-165
overview of, 164–165
单元级测试工具,165–168
unit-level test tools, 165–168
Web 服务测试工具,170
web services test tool, 170
巴赫,乔纳森,201
Bach, Jonathan, 201
后端测试
Back-end testing
在 GUI 后面,282
behind the GUI, 282
非 UI 测试,204–205
non-UI testing, 204–205
竹子,126
Bamboo, 126
采用敏捷开发的障碍,44–49
Barriers to adopting agile development, 44–49
冲突或多重角色,45
conflicting or multiple roles, 45
角色之间的文化差异,48–49
cultural differences among roles, 48–49
缺乏培训,45
lack of training, 45
缺乏对敏捷概念的理解,45–48
lack of understanding of agile concepts, 45–48
失去身份,44-45
loss of identity, 44–45
概述, 44
overview of, 44
过去的经验和态度,48
past experience and attitudes, 48
基线
Baselines
断裂测试基线技术,363
break-test baseline technique, 363
性能,235–237
performance, 235–237
批
Batch
档案, 251
files, 251
处理, 345
processing, 345
调度过程,182
scheduling process, 182
BDD(行为驱动开发)
BDD (Behavior-driven development)
easyb 工具,166–168
easyb tool, 166–168
象限 1 测试工具,127
tools for Quadrant 1 tests, 127
贝南德,马克,51 岁
Benander, Mark, 51
基准测试,237
Benchmarking, 237
Berczuk, 斯蒂芬, 124
Berczuk, Stephen, 124
Beta 测试,466–467
Beta testing, 466–467
大图
Big picture
敏捷测试人员专注于,23
agile testers focus on, 23
高级测试和示例,397–402
high-level tests and examples, 397–402
关键成功因素,490–491
key success factors, 490–491
遗忘的危险,148
peril of forgetting, 148
回归测试和,434
regression tests and, 434
迈克尔·博尔顿,195 岁
Bolton, Michael, 195
博斯,埃里克,114
Bos, Erik, 114
边界条件
Boundary conditions
API测试和,205
API testing and, 205
自动化和, 11
automation and, 11
数据生成工具和,304
data generation tools and, 304
识别测试变体,410
identifying test variations, 410
为137编写测试用例
writing test cases for, 137
博耶、埃里卡、140、163、372、432 _ _ _
Boyer, Erika, 140, 163, 372, 432
头脑风暴
Brainstorming
自动化让测试人员更好地工作,260
automation giving testers better work, 260
象限作为框架,253
quadrants as framework for, 253
花时间,301
taking time for, 301
测试人员,121
testers, 121
断裂测试基线技术,363
Break-test baseline technique, 363
浏览器、兼容性测试和230
Browsers, compatibility testing and, 230
预算限制,55
Budget limits, 55
错误跟踪。请参见 缺陷跟踪
Bug tracking. See Defect tracking
错误。查看 缺陷
Bugs. See Defects
建造
Build
自动化,280–282
automating, 280–282
具有挑战性的候选发布版本,473
challenging release candidate builds, 473
定义,501
definition, 501
增量,178–179
incremental, 178–179
加速,118–119
speeding up, 118–119
Build automation tools, 126, 282
构建/操作/检查模式,180
Build/Operate/Check pattern, 180
构建工具,126
Build tools, 126
构建节拍,126
BuildBeat, 126
业务分析师,374
Business analysts, 374
业务专家角色
Business expert role
agreement regarding requirements, 428, 430
common language and, 134, 291, 414
在客户团队中,6-7
on customer team, 6–7
迭代演示和,443
iteration demo and, 443
291的语言
language of, 291
三的幂和,482
Power of Three and, 482
适合的工具,134
tools geared to, 134
敏捷测试,6
agile testing as, 6
象限 2 & 3, 97–98
Quadrants 2 & 3, 97–98
与120相比,面向技术的测试
technology-facing tests compared with, 120
面向业务的测试,批评产品(第 3 象限),189–215
Business-facing tests, critiquing the product (Quadrant 3), 189–215
验收测试,245
acceptance tests, 245
API 测试,205–206
API testing, 205–206
示威,191–192
demonstrations, 191–192
仿真器工具,213–214
emulator tools, 213–214
端到端测试,249–250
end-to-end tests, 249–250
exploratory testing, 195–202, 248–249
生成测试数据,212
generating test data, 212
GUI 测试,204
GUI testing, 204
监控工具,212-213
monitoring tools, 212–213
概述,189-191
overview of, 189–191
报告,208–210
reports, 208–210
场景测试,192–195
scenario testing, 192–195
基于会话的测试,200-201
session-based testing, 200–201
设置测试,211-212
setting up tests, 211–212
模拟器工具,213
simulator tools, 213
探索性测试工具,210–211
tools for exploratory testing, 210–211
可用性测试,202–204
usability testing, 202–204
用户验收测试,250
user acceptance testing, 250
用户文档,207-208
user documentation, 207–208
Web 服务测试,207
web services testing, 207
面向业务的测试,支持团队(象限 2),129–151
Business-facing tests, supporting team (Quadrant 2), 129–151
提高清晰度,140–142
advance clarity, 140–142
自动化功能测试,245–247
automating functional tests, 245–247
共同语言和,134-135
common language and, 134–135
满意的条件,142-143
conditions of satisfaction and, 142–143
熟度,146–147
doneness, 146–147
推动发展,129–132
driving development with, 129–132
引出要求,135–140
eliciting requirements, 135–140
嵌入式测试,248
embedded testing, 248
渐进方法,144-146
incremental approach, 144–146
需求窘境,132-134
requirements quandary and, 132–134
涟漪效应,143–144
ripple effects, 143–144
风险缓解和147–149
risk mitigation and, 147–149
可测试性和自动化,149–150
testability and automation, 149–150
的工具包。见 工具包(象限 2)
toolkit for. See Toolkit (Quadrant 2)
Web 服务测试,247–248
web services testing, 247–248
业务影响,475–476
Business impact, 475–476
商业价值
Business value
增加价值,31–33
adding value, 31–33
as goal of agile development, 5–8, 69, 454
指标和,75
metrics and, 75
发布周期和3
release cycles and, 3
角色、功能、商业价值模式, 155
role, function, business value pattern, 155
团队方法和16
team approach and, 16
布塞、迈克、106、235、284、313 _ _ _
Busse, Mike, 106, 235, 284, 313
布瓦尔达,汉斯,193 岁
Buwalda, Hans, 193
规范数据、自动化数据库和308–309
Canonical data, automating databases and, 308–309
Canoo Web测试
Canoo WebTest
automating GUI tests, 184, 186
GUI 回归测试套件,291
GUI regression test suite, 291
GUI 冒烟测试,300
GUI smoke tests, 300
GUI 测试工具,174–175
GUI test tools, 174–175
组织测试和,320
organizing tests and, 320
脚本和320
scripts and, 320
XML 编辑器,125
XML Editor for, 125
能力成熟度模型集成 (CMMI),90–91
Capability Maturity Model Integration (CMMI), 90–91
抓拍回放工具,267
Capture-playback tool, 267
庆祝成功
Celebrating successes
改变实施,50-52
change implementation and, 50–52
迭代总结和,449-451
iteration wrap up and, 449–451
钱德拉,阿普瓦,377
Chandra, Apurva, 377
张泰,53–54 岁
Chang, Tae, 53–54
改变
Change
庆祝成功,50-52
celebrating successes, 50–52
赋予团队所有权,50
giving team ownership, 50
介绍, 49
introducing, 49
来之不易,56–57
not coming easy, 56–57
responsiveness to, 28–29
谈论恐惧,49–50
talking about fears, 49–50
清单
Checklists
发布准备,474
release readiness, 474
引出示例和要求的工具,156
tools for eliciting examples and requirements, 156
CI。请参阅 持续集成(CI)
CI. See Continuous integration (CI)
CI厂,126
CI Factory, 126
CMMI(能力成熟度模型集成),90–91
CMMI (Capability Maturity Model Integration), 90–91
同地办公、团队后勤和65–66
Co-location, team logistics and, 65–66
教练员
Coaches
适应敏捷文化,40
adjusting to agile culture and, 40
学习曲线和,266
learning curve and, 266
提供鼓励,69
providing encouragement, 69
技能发展和122
skill development and, 122
培训和,45-46
training and, 45–46
科伯恩,阿利斯泰尔,115
Cockburn, Alistair, 115
代码
Code
自动化和代码通量,269
automation and code flux, 269
自动化和遗留代码,269
automation and legacy code, 269
自动化策略和,303-304
automation strategy and, 303–304
251的文档
documentation of, 251
标准,227
standards, 227
写作可测试,115
writing testable, 115
代码覆盖率、发布指标,360–364
Code coverage, release metrics, 360–364
编码和测试,405–441
Coding and testing, 405–441
增加复杂性,407
adding complexity, 407
处理错误的替代方法,424–428
alternatives for dealing with bugs, 424–428
选择何时修复错误,421–423
choosing when to fix bugs, 421–423
与程序员合作,413–414
collaborating with programmers, 413–414
处理错误,416–419
dealing with bugs, 416–419
决定记录哪些错误,420–421
deciding which bugs to log, 420–421
推动发展,406
driving development and, 406
促进沟通,429-432
facilitating communication, 429–432
专注于一个故事,411-412
focusing on one story, 411–412
识别变异,410
identifying variations, 410
迭代指标,435–440
iteration metrics, 435–440
用于记录错误的媒体,423–424
media for logging bugs, 423–424
概述, 405
overview of, 405
解决观点分歧的三的力量,411
Power of Three for resolving differences in viewpoint, 411
回归测试和432–434
regression testing and, 432–434
资源,434–435
resources, 434–435
风险评估,407-409
risk assessment, 407–409
作为同步过程,409-410,488-489
as simultaneous process, 409–410, 488–489
从简单开始,406、428–429
与客户交谈,414–415
talking to customers, 414–415
批评产品的测试,412–413
tests that critique the product, 412–413
科恩、迈克,50 岁、155 岁、276 岁、296岁、331 岁、332 岁
Cohn, Mike, 50, 155, 276, 296, 331, 332
合作
Collaboration
与客户,396-397
with customers, 396–397
关键成功因素,489–490
key success factors, 489–490
与程序员,413-414
with programmers, 413–414
整个团队的方法,15-16
whole team approach, 15–16
沟通
Communication
共同语言和,134-135
common language and, 134–135
DTS(缺陷跟踪系统)和,83
DTS (Defect Tracking System) and, 83
产品交付和462–463
product delivery and, 462–463
尺寸作为挑战,42-43
size as challenge to, 42–43
团队之间,69–70
between teams, 69–70
测试结果,357–358
test results, 357–358
比较,自动化,283
Comparisons, automating, 283
兼容性测试,229–230
Compatibility testing, 229–230
组件测试
Component tests
自动化, 282
automating, 282
定义,501
definition, 501
支持功能,5
supporting function of, 5
满意条件
Conditions of satisfaction
面向业务的测试,以及142–143
business-facing tests and, 142–143
定义,501–502
definition, 501–502
上下文驱动测试
Context-driven testing
定义,502
definition, 502
象限和106–107
quadrants and, 106–107
持续构建过程
Continuous build process
失败通知和112
failure notification and, 112
反馈和,119
feedback and, 119
FitNesse 测试和357
FitNesse tests and, 357
实施,114
implementing, 114
integrating tools with, 175, 311
源代码控制和124
source code control and, 124
测试人员可以做什么,121
what testers can do, 121
持续反馈原则,22
Continuous feedback principle, 22
持续改进原则,27-28
Continuous improvement principle, 27–28
持续集成 (CI)
Continuous integration (CI)
自动化,280–282
automating, 280–282
作为核心实践,486–487
as core practice, 486–487
可安装性和231–232
installability and, 231–232
远程数据监控系统示例,244
Remote Data Monitoring system example, 244
运行测试和,111–112
running tests and, 111–112
转换、数据迁移和460–461
Conversion, data migration and, 460–461
核心实践
Core practices
编码和测试作为一个过程,488–489
coding and testing as one process, 488–489
持续集成,486–487
continuous integration, 486–487
渐进方法,488
incremental approach, 488
概述, 486
overview of, 486
实践之间的协同作用,489
synergy between practices, 489
技术债务管理,487–488
technical debt management, 487–488
测试环境,487
test environments, 487
Courage, principles, 25–26, 71
信誉建设,57
Credibility, building, 57
批评产品
Critiquing the product
企业面临考验。请参阅 面向业务的测试,评价产品(第 3 象限)
business facing tests. See Business-facing tests, critiquing the product (Quadrant 3)
面向技术的测试。请参阅 面向技术的测试,批评产品(象限 4)
technology-facing tests. See Technology-facing tests, critiquing the product (Quadrant 4)
CrossCheck,测试 Web 服务,170
CrossCheck, testing Web Services, 170
Cultural change, 37. See also Organizations
Cunningham, Ward, 106, 168, 506
客户期望
Customer expectations
业务影响和475–476
business impact and, 475–476
生产支持,475
production support, 475
面向客户的测试。查看 面向业务的测试
Customer-facing test. See Business-facing tests
客户支持、DTS(缺陷跟踪系统)和82
Customer support, DTS (Defect Tracking System) and, 82
客户团队
Customer team
定义,502
definition, 502
客户和开发团队之间的互动,8
interaction between customer and developer teams, 8
概述,7
overview of, 7
客户测试
Customer testing
Alpha/Beta 测试,466–467
Alpha/Beta testing, 466–467
定义,502
definition, 502
概述, 464
overview of, 464
UAT(用户验收测试),464–466
UAT (user acceptance testing), 464–466
顾客
Customers
collaborating with, 396–397, 489–490
在迭代规划期间考虑所有观点,388–389
considering all viewpoints during iteration planning, 388–389
delivering value to, 22–23
importance of communicating with, 140, 414–415, 444
iteration demo, 191–192, 443–444
参与迭代计划,384–385
participation in iteration planning,384–385
关系, 41–42
relationship with, 41–42
审查高级测试,400
reviewing high-level tests with, 400
用同一个声音说话,373–374
speaking with one voice, 373–374
CVS、源代码控制和124
CVS, source code control and, 124
数据
Data
自动创建或设置,284–285
automating creation or setup, 284–285
清理, 461
cleanup, 461
转换,459–461
conversion, 459–461
发布计划和348
release planning and, 348
写任务卡和,392
writing task cards and, 392
数据驱动测试,182–183
Data-driven tests, 182–183
数据馈送、测试、249
Data feeds, testing, 249
数据生成工具,304-305
Data generation tools, 304–305
Data migration, automating, 310, 460
数据库
Databases
运行测试时避免访问,306–310
avoiding access when running tests, 306–310
规范数据和自动化,308–309
canonical data and automation, 308–309
可维护性和,228
maintainability and, 228
产品交付和更新,459–461
product delivery and updates, 459–461
类似生产的数据和自动化,309–310
production-like data and automation, 309–310
为每个自动化测试设置/拆除数据,307–308
setting up/tearing down data for each automated test, 307–308
测试数据迁移,310
testing data migration, 310
德索萨,肯,223
De Souza, Ken, 223
截止日期、范围和340–341
Deadlines, scope and, 340–341
缺陷指标
Defect metrics
概述,437–440
overview of, 437–440
发布指标,364–366
release metrics, 364–366
缺陷跟踪,79-86
Defect tracking, 79–86
DTS(缺陷跟踪系统),79-83
DTS (Defect Tracking System), 79–83
保持专注,85–86
keeping focus and, 85–86
概述, 79
overview of, 79
原因,79
reasons for, 79
83–85的工具
tools for, 83–85
缺陷跟踪系统。参见 DTS(缺陷跟踪系统)
Defect Tracking System. See DTS (Defect Tracking System)
处理错误的替代方法,424–428
alternatives for dealing with bugs,424–428
选择何时修复错误,421–423
choosing when to fix bugs, 421–423
处理错误,416–419
dealing with bugs, 416–419
决定记录哪些错误,420–421
deciding which bugs to log, 420–421
用于记录错误的媒体,423–424
media for logging bugs, 423–424
指标和,79
metrics and, 79
TDD(测试驱动开发)和,490
TDD (test-driven development) and, 490
写任务卡和,391-392
writing task cards and, 391–392
zero bug tolerance, 79, 418–419
可交付成果
Deliverables
“安装并完成”可交付成果,454
“fit and finish” deliverables, 454
非软件,470
nonsoftware, 470
概述,468–470
overview of, 468–470
Alpha/Beta 测试,466–467
Alpha/Beta testing, 466–467
业务影响和475–476
business impact and, 475–476
通讯和,462-463
communication and, 462–463
客户期望,475
customer expectations, 475
客户测试,464
customer testing, 464
数据转换和数据库更新,459–461
data conversion and database updates, 459–461
可交付成果,468–470
deliverables, 468–470
游戏结束,456–457
end game, 456–457
安装测试,461–462
installation testing, 461–462
与外部应用程序集成,459
integration with external applications, 459
非功能性测试和458–459
nonfunctional testing and, 458–459
概述, 453
overview of, 453
包装,474–475
packaging, 474–475
计划测试时间,455–456
planning time for testing, 455–456
开发后测试周期,467–468
post-development testing cycles, 467–468
生产支持,475
production support, 475
发布验收标准,470–473
release acceptance criteria, 470–473
发布产品,470
releasing product, 470
暂存环境和,458
staging environment and, 458
测试候选版本,458
testing release candidates, 458
UAT(用户验收测试),464–466
UAT (user acceptance testing), 464–466
如果还没有准备好怎么办,463–464
what if it is not ready, 463–464
是什么造就了产品,453–455
what makes a product, 453–455
演示/示范
Demos/demonstrations
迭代次数,443–444
of an iteration, 443–444
对客户的价值,191–192
value to customers, 191–192
部署、自动化,280–282
Deployment, automating, 280–282
设计
Design
自动化策略和,292–294
automation strategy and, 292–294
设计时考虑到测试,115–118
designing with testing in mind, 115–118
详细的测试用例
Detailed test cases
写作艺术与科学,178
art and science of writing, 178
大局观和,148-149
big picture approach and, 148–149
设计,401
designing with, 401
开发团队
Developer team
客户和开发团队之间的互动,8
interaction between customer and developer teams, 8
概述,7-8
overview of, 7–8
发展
Development
自动驾驶测试,262–263
automated tests driving, 262–263
面向业务的驾驶测试,129–132
business-facing tests driving, 129–132
编码驾驶,406
coding driving, 406
开发后测试周期,467–468
post-development testing cycles, 467–468
发展高峰,381
Development spikes, 381
开发团队,502
Development team, 502
差异工具,283
diff tool, 283
分布式团队,431–432
Distributed teams, 431–432
缺陷追踪系统,82
defect tracing systems, and, 82
实体物流,66
physical logistics, 66
在线高级测试,399
online high level tests for, 399
在线故事板,357
online story board for, 357
应对变化,29
responding to change, 29
基于软件的工具来引出示例和需求,以及163–164
software-based tools to elicit examples and requirements, and, 163–164
文档
Documentation
自动化测试作为来源,263–264
automated tests as source of, 263–264
问题和修复,417
problems and fixes, 417
报告,208–210
reports, 208–210
测试代码,251
of test code, 251
测试为,402
tests as, 402
用户文档,207-208
user documentation, 207–208
熟度
Doneness
知道故事何时结束,104–105
knowing when a story is done, 104–105
多层, 471–472
multitiered, 471–472
通过测试推动开发。请参见TDD(测试驱动开发)
Driving development with tests. See TDD (test-driven development)
DTS(缺陷跟踪系统),80-83
DTS (Defect Tracking System), 80–83
80-82的好处
benefits of, 80–82
选择用于记录错误的媒体,424
choosing media for logging bugs, 424
记录问题和修复,417
documenting problems and fixes, 417
记录错误和420
logging bugs and, 420
不使用的原因,82-83
reason for not using, 82–83
戴蒙德、罗宾、xxx
Dymond, Robin, xxx
动态分析,安全测试工具,225
Dynamic analysis, security testing tools, 225
easyb 行为驱动开发工具,165–168
easyb behavior-driven development tool, 165–168
轻松模拟,127
EasyMock, 127
边缘案例
Edge cases
识别变异,410
identifying variations, 410
没时间,112
not having time for, 112
从简单开始,然后增加复杂性,406–407
starting simple and then adding complexity, 406–407
测试用例,137
test cases for, 137
嵌入式系统,远程数据监控实例,248
Embedded system, Remote Data Monitoring example, 248
赋权,团队,44
Empowerment, of teams, 44
仿真器工具,213–214
Emulator tools, 213–214
结束游戏
End game
敏捷测试,91
Agile testing, 91
迭代,14
iteration, 14
产品交付和456–457
product delivery and, 456–457
释放和,327
release and, 327
端到端测试,249–250
End-to-end tests, 249–250
享受,原则,31
Enjoyment, principle of, 31
环境,测试环境,347-348
Environment, test environment, 347–348
史诗。另见 主题
Epic. See also Themes
定义,502
definition, 502
功能成为,502
features becoming, 502
规划, 252
planning, 252
ePlan Services, Inc., xli, 267
ePlan Services, Inc., xli, 267
错误,手动测试和,259
Errors, manual testing and, 259
估计故事大小,332–338
Estimating story size, 332–338
有效, 234
eValid, 234
基于事件的模式,测试设计模式,181
Event-based patterns, test design patterns, 181
为团队、测试人员和您编写 Ruby 日常脚本(Marick), 297 , 303
Everyday Scripting with Ruby for Teams, Testers, and You (Marick), 297, 303
示例驱动开发,378–380
Example-driven development, 378–380
例子
Examples
用于引发需求,136-137
for eliciting requirements, 136–137
引出示例和要求的工具,155-156
tools for eliciting examples and requirements, 155–156
可执行测试,406
Executable tests, 406
探索性测试(ET)
Exploratory testing (ET)
活动、特征和技能(夏甲),198–200
activities, characteristics, and skills (Hagar), 198–200
探索性测试者的属性,201–202
attributes of exploratory tester, 201–202
自动化,201
automation of, 201
定义,502–503
definition, 502–503
结束游戏,457
end game and, 457
解释(博尔顿),195-198
explained (Bolton), 195–198
手动测试和,280
manual testing and, 280
监控工具, 212
monitoring tools, 212
远程数据监控系统示例,248–249
Remote Data Monitoring system example, 248–249
基于会话的测试和200–201
session-based testing and, 200–201
设置,211–212
setup, 211–212
模拟器和模拟器,212-213
simulators and emulators, 212–213
批评产品的测试,412–413
tests that critique the product, 412–413
工具,210–212
tools for, 210–212
用于生成测试数据的工具,212
tools for generating test data, 212
什么不应该自动化,286
what should not be automated, 286
External quality, business facing tests defining, 99, 131
极限编程。参见 XP(极限编程)
Extreme Programming. See XP (Extreme Programming)
极限编程解释(Beck),26
Extreme Programming Explained (Beck), 26
面对面交流,23-25
Face-to-face communication, 23–25
故障转移测试,232
Failover tests, 232
失败,勇于学习,25
Failure, courage to learn from, 25
Fake objects, 115, 118, 306, 502–503
容错、产品交付和、459
Fault tolerance, product delivery and, 459
害怕
Fear
自动化的障碍,269–270
barriers to automation, 269–270
改变,49-50
change and, 49–50
无所畏惧的变化(Manns 和 Rising),121
Fearless Change (Manns and Rising), 121
特征
Features
缺陷对比,417–418
defects vs., 417–418
定义,502–503
definition, 502–503
关注价值,341
focusing on value, 341
自动化测试提供,262
automated tests providing, 262
持续反馈原则,22
continuous feedback principle, 22
迭代方法和,299-300
iterative approach and, 299–300
关键成功因素,484–486
key success factors, 484–486
管理测试,323–324
managing tests for, 323–324
象限 1 测试和,118–119
Quadrant 1 tests and, 118–119
“安装并完成”可交付成果,454
“Fit and finish” deliverables, 454
Fit(集成测试框架),134–135
Fit (Framework for Integrated Test), 134–135
API 层功能测试工具,168–169
API-layer functional test tools, 168–169
自动化测试金字塔和,278
automation test pyramid and, 278
FIT IssueTrack,Alcea,83–84
FIT IssueTrack, Alcea, 83–84
健身
FitNesse
163的优势
advantages of, 163
API 层功能测试工具,169–170
API-layer functional test tools, 169–170
automating functional tests with, 30, 145
business-facing tests with, 154, 178
协作,164
collaboration and, 164
continual builds and, 119, 357
数据验证,287
data verification with, 287
熟度和,472
doneness and, 472
鼓励使用,122
encouraging use of, 122
反馈和323–324
feedback and, 323–324
文件解析规则说明,205
file parsing rules illustrated with, 205
functional testing behind the GUI, 291, 300
本土脚本和305
home-grown scripts and, 305
与 JUnit 相比,299
JUnit compared with, 299
用于自动化测试的关键字或操作词,182–183
keywords or actions words for automating tests, 182–183
手动与自动测试,210
manual vs. automated testing, 210
内存需求,306
memory demands of, 306
组织测试和319–320
organizing tests and, 319–320
概述,168–170
overview of, 168–170
远程测试和,432
remote testing and, 432
“开始、停止、继续”列表,446
“start, stop, continue” list, 446
支持源代码控制工具,320
support for source code control tools, 320
测试自动化金字塔和,278
test automation pyramid and, 278
测试卡和,389–390
test cards and, 389–390
测试用例作为文档,402
test cases as documentation, 402
测试设计和维护,292
test design and maintenance, 292
测试数据库层,284
testing database layer with, 284
测试故事,395
testing stories, 395
可追溯性要求和88
traceability requirements and, 88
用户验收测试,295
user acceptance testing, 295
维基和186
wikis and, 186
流程图
Flow diagrams
场景测试和194–195
scenario testing and, 194–195
引出示例和要求的工具,160-163
tools for eliciting examples and requirements, 160–163
马丁·福勒,117
Fowler, Martin, 117
集成测试框架。See Fit(集成测试框架)
Framework for Integrated Test. See Fit (Framework for Integrated Test)
框架,90-93
Frameworks, 90–93
ftptt, 234
ftptt, 234
职能分析师,386
Functional analysts, 386
兼容性问题和230
compatibility issues and, 230
定义,502–503
definition, 502–503
端到端测试,249–250
end-to-end tests, 249–250
层数,246
layers, 246
与225相比,非功能测试
nonfunctional tests compared with, 225
远程数据监控系统示例,245–247
Remote Data Monitoring system example, 245–247
地域分散的团队
Geographically dispersed teams
应对,376–378
coping with, 376–378
促进沟通,431-432
facilitating communication and, 431–432
安德鲁·格洛弗,166 岁
Glover, Andrew, 166
绿地项目
Greenfield projects
代码测试和,116
code testing and, 116
定义,502–503
definition, 502–503
GUI(图形用户界面)
GUI (graphical user interface)
自动化策略和,293
automation strategy and, 293
代码通量和,269
code flux and, 269
标准,227
standards, 227
GUI 冒烟测试
GUI smoke tests
Canoo WebTest 和300
Canoo WebTest and, 300
持续构建,119
continual builds and, 119
缺陷指标,437
defect metrics, 437
GUI 测试工具,170–176
GUI test tools, 170–176
Canoo 网络测试,174–175
Canoo Web Test, 174–175
“自制”测试自动化工具,175
“home-brewed” test automation tools, 175
开源测试工具,172
open source test tools, 172
概述,170-171
overview of, 170–171
记录/回放工具,171–172
record/playback tools, 171–172
红宝石与 Watir,172–174
Ruby with Watir, 172–174
硒,174
Selenium, 174
API 测试,205–206
API testing, 205–206
自动化测试金字塔和,278
automation test pyramid and, 278
GUI冒烟测试,119、300、437 _
GUI smoke tests, 119, 300, 437
概述, 204
overview of, 204
Web 服务测试,207
Web service testing, 207
乔恩·夏甲,198 岁
Hagar, Jon, 198
硬件
Hardware
兼容性和229
compatibility and, 229
测试环境的成本,487
cost of test environments, 487
功能测试和,230
functional testing and, 230
投资自动化,267
investing in automation and, 267
生产环境和,310
production environment and, 310
可扩展性和,233
scalability and, 233
测试基础设施,319
test infrastructure, 319
测试产品安装,462
testing product installation, 462
亨德里克森,伊丽莎白,203,315–316
Hendrickson, Elisabeth, 203, 315–316
高级测试用例,397–402
High-level test cases, 397–402
模型,398–399
mockups, 398–399
概述,397–398
overview of, 397–398
与客户一起审查,400
reviewing with customers, 400
与程序员一起审查,400–401
reviewing with programmers, 400–401
测试用例作为文档,402
test cases as documentation, 402
雇用一名测试员,67–69 岁
Hiring a tester, 67–69
自主研发的测试工具
Home-grown test tool
自动化工具, 314
automation tools, 314
GUI 测试工具,175
GUI test tools, 175
测试结果,323
test results, 323
HTTPerf,234
httperf, 234
哈德森,126 岁
Hudson, 126
IBM Rational ClearCase,124
IBM Rational ClearCase, 124
IDEs (Integrated Development Environments)
定义,502–503
definition, 502–503
日志分析工具,212
log analysis tools, 212
象限 1 测试的工具,124–126
tools for Quadrant 1 tests, 124–126
“能力”测试
“ility” testing
兼容性测试,229–230
compatibility testing, 229–230
安装性测试,231-232
installability testing, 231–232
互操作性测试,228–229
interoperability testing, 228–229
可维护性测试,227-228
maintainability testing, 227–228
reliability testing, 230–231, 250–251
安全测试,223-227
security testing, 223–227
影响,全系统,342
Impact, system-wide, 342
实施精益软件开发:从概念到现金( Poppendieck),74,416
Implementing Lean Software Development: From Concept to Cash (Poppendieck), 74, 416
改进
Improvement
流程改进方法,448–449
approach to process improvement, 448–449
持续改进原则,27-28
continuous improvement principle, 27–28
从回顾中改进的想法,447-449
ideas for improvement from retrospectives, 447–449
增量开发
Incremental development
逐步构建测试,178–179
building tests incrementally, 178–179
作为核心实践,488
as core practice, 488
“能力”测试和232
“ilities” tests and, 232
薄片,小块,144-146
thin slices, small chunks, 144–146
传统与敏捷测试,12–13
traditional vs. agile testing, 12–13
索引卡,记录错误,423
Index cards, logging bugs on, 423
基础设施
Infrastructure
象限 1 测试,111–112
Quadrant 1 tests, 111–112
测试基础设施,319
test infrastructure, 319
测试计划和346–347
test plans and, 346–347
安装性测试,231-232
Installability testing, 231–232
安装测试,461–462
Installation testing, 461–462
Integrated Development Environments. See IDEs (Integrated Development Environments)
集成测试
Integration testing
互操作性和229
interoperability and, 229
产品和外部应用程序,459
product and external applications, 459
IntelliJ IDEA,125
IntelliJ IDEA, 125
内在品质
Internal quality
衡量代码的内部质量,99
measuring internal quality of code, 99
满足团队标准,366
meeting team standards, 366
象限 1 测试和,111
Quadrant 1 tests and, 111
速度和,112
speed and, 112
互操作性测试,228–229
Interoperability testing, 228–229
投资、自动化要求,267–268
Investment, automation requiring, 267–268
迭代
Iteration
自动化策略和,299–300
automation strategy and, 299–300
定义,502–503
definition, 502–503
演示,443–444
demo, 443–444
测试仪的寿命和327
life of a tester and, 327
预迭代活动。请参阅 预迭代活动
pre-iteration activities. See Pre-iteration activities
优先报道,338
prioritizing stories and, 338
传统与敏捷测试,12–13
traditional vs. agile testing, 12–13
迭代开始,383–403
Iteration kickoff, 383–403
与客户合作,396–397
collaboration with customers, 396–397
考虑所有观点,385-389
considering all viewpoints, 385–389
控制工作量,393
controlling workload, 393
高级测试和示例,397–402
high-level tests and examples, 397–402
迭代计划,383–384
iteration planning, 383–384
学习项目细节,384–385
learning project details, 384–385
概述, 383
overview of, 383
可测试的故事,393–396
testable stories, 393–396
写任务卡,389–392
writing task cards, 389–392
迭代指标,435–440
Iteration metrics, 435–440
缺陷指标,437–440
defect metrics, 437–440
测量进度,435–437
measuring progress with, 435–437
概述, 435
overview of, 435
有用性,439–440
usefulness of, 439–440
迭代计划
Iteration planning
考虑所有观点,385-389
considering all viewpoints, 385–389
控制工作量,393
controlling workload, 393
学习项目细节,384–385
learning project details, 384–385
概述, 383–384
overview of, 383–384
写任务卡,389–392
writing task cards, 389–392
迭代评审会议,415
Iteration review meeting, 415
迭代结束,443–451
Iteration wrap up, 443–451
庆祝成功,449–451
celebrating successes, 449–451
迭代演示,443–444
demo of iteration, 443–444
改进的想法,447-449
ideas for improvement, 447–449
回顾,444-445
retrospectives, 444–445
回顾的“开始、停止、继续”练习,445–447
“start, stop, continue” exercise for retrospectives, 445–447
ITIL(信息技术基础设施库),90-91
ITIL (Information Technology Infrastructure Library), 90–91
行为举止,165
JBehave, 165
J控制台,234
JConsole, 234
性能基线测试,235
performance baseline tests, 235
performance testing, 223, 234, 313
JMS(Java 消息服务)
JMS (Java Messaging Service)
定义,502–503
definition, 502–503
与外部应用程序集成,243
integration with external applications and, 243
测试数据馈送和249
testing data feeds and, 249
J探查器,234
JProfiler, 234
J单元
JUnit
FitNesse 作为 TDD 的替代品,299
FitNesse as alternative for TDD, 299
功能测试,176
functional testing, 176
负载测试工具,234–235
load testing tools, 234–235
单元测试工具,126、165、291 _
unit test tools, 126, 165, 291
JUnitPerf,234
JUnitPerf, 234
Just in time development, 369. See also Pre-iteration activities
关键成功因素
Key success factors
敏捷测试心态,482–483
agile testing mind-set, 482–483
自动化回归测试,484
automating regression testing, 484
大局观,490–491
big picture approach, 490–491
编码和测试作为一个过程,488–489
coding and testing as one process, 488–489
与客户合作,489–490
collaboration with customers, 489–490
持续集成 (CI),486–487
continuous integration (CI), 486–487
反馈,484–486
feedback, 484–486
核心实践基础,486
foundation of core practices, 486
增量方法(薄片,小块),488
incremental approach (thin slices, small chunks), 488
概述, 481
overview of, 481
实践之间的协同作用,489
synergy between practices, 489
技术债务管理,487–488
technical debt management, 487–488
测试环境,487
test environments, 487
整个团队的方法,482
whole team approach, 482
关键字驱动的测试,182–183
Keyword-driven tests, 182–183
国王,约瑟夫,176 岁
King, Joseph, 176
知识库,DTS,80-81
Knowledge base, DTS, 80–81
König, Dierk, 320
König, Dierk, 320
语言,共同需求,134–135
Language, need for common, 134–135
分层架构,116
Layered architecture, 116
精益测量,指标,74-75
Lean measurements, metrics, 74–75
学习
Learning
自动化策略和,303
automation strategy and, 303
持续改进原则,27
continuous improvement principle, 27
Learning curve, automation and, 266–267, 303
遗留代码,269
Legacy code, 269
遗留代码救援(Feathers),117
Legacy code rescue (Feathers), 117
遗留系统
Legacy systems
CCDE,269
ccde, 269
定义,502–503
definition, 502–503
记录错误和421
logging bugs and, 421
测试, 117
testing, 117
软件测试中的经验教训(Pettichord),485
Lessons Learned in Software Testing (Pettichord), 485
Lessons learned sessions, 383. See also Retrospectives
轻量级流程,73–74
Lightweight processes, 73–74
轻量级测试计划,350
Lightweight test plans, 350
负载测试。请参见 性能和负载测试
Load testing. See Performance and load testing
负载运行器,234
LoadRunner, 234
负载测试,234
LoadTest, 234
物流,物理,65–66
Logistics, physical, 65–66
LogWatch 工具,212
LogWatch tool, 212
失去身份,QA 团队感到恐惧,44–45
Loss of identity, QA teams fearing, 44–45
卢维翁,克里斯托夫,63 岁
Louvion, Christophe, 63
可维护性测试,227–228
Maintainability testing, 227–228
管理,52–55
Management, 52–55
提高清晰度,373–374
advance clarity and, 373–374
文化变革,52–54
cultural change and, 52–54
概述, 52
overview of, 52
providing metrics to, 440
管理人员
Managers
52-54 岁的文化变革
cultural changes for, 52–54
如何影响测试,122–123
how to influence testing, 122–123
说经理的语言,55
speaking managerís language, 55
曼斯,玛丽·林恩,121–122
Manns, Mary Lynn, 121–122
手动测试
Manual testing
自动化与258–259
automation vs., 258–259
危险的, 289
peril of, 289
马里克、布赖恩、5、24、97、134、170、203、303 _ _ _ _ _ _ _ _ _
Marick, Brian, 5, 24, 97, 134, 170, 203, 303
马丁·迈卡,169
Martin, Micah, 169
马丁·罗伯特·C.,169
Martin, Robert C., 169
矩阵
Matrices
高级测试和398–399
high-level tests and, 398–399
文本矩阵,350–353
text matrices, 350–353
马文,126 岁
Maven, 126
麦克马洪,克里斯,260
McMahon, Chris, 260
平均无故障时间,可靠性测试,230
Mean time between failure, reliability testing, 230
平均无故障时间,可靠性测试,230
Mean time to failure, reliability testing, 230
媒体,用于记录错误,423–424
Media, for logging bugs, 423–424
会议
Meetings
地理位置分散,376
geographically dispersed, 376
迭代开始,372
iteration kickoff, 372
迭代计划, 23–24 , 244 , 331 , 384 , 389
iteration planning, 23–24, 244, 331, 384, 389
预先规划,370–372
pre-planning, 370–372
回顾, 447
retrospective, 447
调度, 70
scheduling, 70
上浆过程和336–337
sizing process and, 336–337
团队参与,32
team participation and, 32
测试计划,263
test planning, 263
内存泄漏,237–238
Memory leaks, 237–238
内存管理测试,237–238
Memory management testing, 237–238
梅萨罗斯、杰拉德、99、111、113、138、146、182、204、291、296、430 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Meszaros, Gerald, 99, 111, 113, 138, 146, 182, 204, 291, 296, 430
指标,74–79
Metrics, 74–79
代码覆盖率,360–364
code coverage, 360–364
77-78的沟通
communication of, 77–78
缺陷指标,364-366,437-440
defect metrics, 364–366, 437–440
迭代指标,435–440
iteration metrics, 435–440
证明自动化投资的合理性,268
justifying investment in automation, 268
精益测量,74-75
lean measurements, 74–75
概述, 74
overview of, 74
通过测试,358–360
passing tests, 358–360
reasons for tracking defects, 52, 75–77, 82
发布指标,358
release metrics, 358
投资回报率和78–79
ROI and, 78–79
不该做什么,77
what not to do with, 77
XP 雷达图,47–48
XP radar charts, 47–48
里程碑,庆祝成功,449–450
Milestones, celebrating successes, 449–450
MIME(多用途 Internet 邮件扩展)
MIME (Multipurpose Internet Mail Extensions)
定义,504
definition, 504
测试数据馈送和249
testing data feeds and, 249
思维导图,156–158
Mind maps, 156–158
心态
Mind-set
敏捷测试作为,20-21
agile testing as, 20–21
关键成功因素,482–483
key success factors, 482–483
主动,369–370
pro-active, 369–370
“迷你瀑布”现象,46-47
“Mini-waterfall” phenomenon, 46–47
模拟对象
Mock objects
定义,504
definition, 504
风险缓解和459
risk alleviation and, 459
实施工具,127
tools for implementing, 127
单元测试和,114
unit tests and, 114
模型
Mock-ups
促进沟通,430
facilitating communication and, 430
高级测试和398–399
high-level tests and, 398–399
故事和,380
stories and, 380
引出示例和要求的工具,160
tools for eliciting examples and requirements, 160
模型驱动开发,398
Model-driven development, 398
楷模
Models
质量模型,90-93
quality models, 90–93
UI 建模示例,399
UI modeling example, 399
Monitoring tools, 212–213, 235
多层方法,自动化策略,290-292
Multi-layered approach, automation strategy, 290–292
多用途 Internet 邮件扩展 (MIME)
Multipurpose Internet Mail Extensions (MIME)
定义,504
definition, 504
测试数据馈送和249
testing data feeds and, 249
命名约定,227
Naming conventions, 227
南特,126 岁
Nant, 126
导航、可用性测试和204
Navigation, usability testing and, 204
N行为,165
NBehave, 165
新负载,234
NeoLoad, 234
Nessus,漏洞扫描器,226
Nessus, vulnerability scanner, 226
.NET 内存分析器,234
.NET Memory Profiler, 234
网豆,125
NetBeans, 125
网络侦察兵,235
NetScout, 235
非功能测试。另见面 向技术的测试,评价产品(象限 4)
Non-functional testing. See also Technology-facing tests, critiquing the product (Quadrant 4)
交付产品和458–459
delivering product and, 458–459
功能测试比较,225
functional testing compared with, 225
要求,218–219
requirements, 218–219
什么时候执行,222
when to perform, 222
北,丹,165
North, Dan, 165
NSpec,165
NSpec, 165
Oleszkiewicz, 雅库布, 418
Oleszkiewicz, Jakub, 418
一次性测试,286–287
One-off tests, 286–287
开源工具
Open source tools
敏捷开源测试工具,172–175
agile open source test tools, 172–175
自动化和,314-315
automation and, 314–315
GUI 测试工具,172
GUI test tools, 172
IDE,124–125
IDEs, 124–125
打开网页加载,234
OpenWebLoad, 234
操作系统 (OS)、兼容性测试和230
Operating systems (OSs), compatibility testing and, 230
组织,37–44
Organizations, 37–44
敏捷开发的挑战,35
challenges of agile development, 35
冲突的文化,43
conflicting cultures, 43
客户关系和41–42
customer relationships and, 41–42
概述,37-38
overview of, 37–38
质量理念,38-40
quality philosophy, 38–40
大小和,42-43
size and, 42–43
可持续的测试速度,40-41
sustainable pace of testing and, 40–41
团队授权,44
team empowerment, 44
OS(操作系统)、兼容性测试和230
OSs (operating systems), compatibility testing and, 230
所有权,赋予团队所有权,50
Ownership, giving team ownership, 50
包装、产品交付和474–475
Packaging, product delivery and, 474–475
结对编程
Pair programming
代码审查和227
code review and, 227
受过培训的开发人员,61
developers trained in, 61
IDE 和125
IDEs and, 125
团队方法和,244
team approach and, 244
配对测试,413
Pair testing, 413
通过测试,发布指标,358–360
Passing tests, release metrics, 358–360
性能监视器,235
PerfMon, 235
Perforce, 124
Perforce, 124
性能和负载测试
Performance and load testing
自动化, 283
automating, 283
基线,235–237
baselines, 235–237
内存管理测试,237–238
memory management testing, 237–238
概述, 234
overview of, 234
产品交付和,458
product delivery and, 458
可扩展性测试,233–234
scalability testing, 233–234
测试环境,237
test environment, 237
工具,234–235
tools for, 234–235
什么时候执行,223
when to perform, 223
谁执行测试,220–221
who performs the test, 220–221
绩效、奖励和70–71
Performance, rewards and, 70–71
危险
Perils
忘记大局,148
forgetting the big picture, 148
素质警察心态,39
quality police mentality, 39
测试紧缩,416
the testing crunch, 416
等待周二的构建,280
waiting for Tuesdayís build, 280
你不是团队的一员,32
youíre not really part of the team, 32
Perl剪辑
PerlClip
数据生成工具,305
data generation tools, 305
用于生成测试数据的工具,212
tools for generating test data, 212
角色测试,202–204
Persona testing, 202–204
佩蒂索德、布雷特、175、264、485 _
Pettichord, Bret, 175, 264, 485
Phased and gated development, 73–74, 129
实体物流,65–66
Physical logistics, 65–66
规划
Planning
前进, 43
advance, 43
迭代。请参见 迭代计划
iteration. See Iteration planning
发布/主题策划。请参阅 发布计划
release/theme planning. See Release planning
测试。请参见 测试计划
testing. See Test planning
PMO(项目管理办公室),440
PMO (Project Management Office), 440
波尔斯,安迪,134
Pols, Andy, 134
端口和适配器模式 (Cockburn),115
Ports and Adapters pattern (Cockburn), 115
开发后测试,467–468
Post-development testing, 467–468
迭代后错误,421
Post-iteration bugs, 421
庞德,234
Pounder, 234
三的力量
Power of Three
业务专家和,482
business expert and, 482
寻找共同语言,430
finding a common language, 430
good communication and, 33, 490
解决问题,24
problem solving and, 24
resolving differences in viewpoint, 401, 411
整个团队的方法和,482
whole team approach and, 482
务实的项目自动化,260
Pragmatic Project Automation, 260
预迭代活动,369–382
Pre-iteration activities, 369–382
提前澄清,373
advance clarity, 373
提前制作故事的好处,370–372
benefits of working on stories in advance, 370–372
客户用同一种声音说话,373–374
customers speaking with one voice, 373–374
确定故事大小,375–376
determining story size, 375–376
评估所需的预先准备量,372–373
evaluating amount of advance preparation needed, 372–373
示例,378–380
examples, 378–380
收集有关需求的所有观点,374-375
gathering all viewpoints regarding requirements, 374–375
地理位置分散的团队,376–378
geographically dispersed team and, 376–378
概述, 369
overview of, 369
优先考虑缺陷,381
prioritizing defects, 381
积极主动的心态,369–370
pro-active mindset, 369–370
资源, 381
resources, 381
测试策略和,380–381
test strategies and, 380–381
预先计划会议,370-372
Pre-planning meeting, 370–372
原理、自动化
Principles, automation
敏捷编码实践,303–304
agile coding practices, 303–304
迭代方法,299–300
iterative approach, 299–300
保持简单,298–299
keep it simple, 298–299
边做边学,303
learning by doing, 303
概述, 298
overview of, 298
花时间做对,301-303
taking time to do it right, 301–303
整个团队方法,300–301
whole team approach, 300–301
原则,适用于敏捷测试人员
Principles, for agile testers
持续反馈,22
continuous feedback, 22
持续改进,27–28
continuous improvement, 27–28
勇气,25–26
courage, 25–26
为客户提供价值,22–23
delivering value to customer, 22–23
享受, 31
enjoyment, 31
面对面交流,23-25
face-to-face communication, 23–25
保持简单,26–27
keeping it simple, 26–27
概述,21-22
overview of, 21–22
以人为本,30
people focus, 30
响应变化,28-29
responsive to change, 28–29
自组织,29-30
self-organizing, 29–30
确定缺陷的优先级,381
Prioritizing defects, 381
优先报道,338-340
Prioritizing stories, 338–340
积极主动的心态,369–370
Pro-active mindset, 369–370
产品
Product
商业价值,31–33
business value, 31–33
送货。请参阅 交付产品
delivery. See Delivering product
批判性测试(Q3 和 Q4),101–104
tests that critique (Q3 & Q4), 101–104
是什么造就了产品,453–455
what makes a product, 453–455
产品拥有者
Product owner
在迭代计划中考虑所有观点,386–389
considering all viewpoints during iteration planning, 386–389
定义,504
definition, 504
迭代计划和,384
iteration planning and, 384
适合的工具,134
tools geared to, 134
生产
Production
记录错误和421
logging bugs and, 421
支持, 475
support, 475
生产代码
Production code
自动化测试金字塔和,277–278
automation test pyramid and, 277–278
定义,504
definition, 504
delivering value to, 70
程序员写作,48
programmers writing, 48
源代码控制和434
source code control and, 434
与测试同步,322
synchronization with testing, 322
测试先行的开发和,113
test-first development and, 113
测试支持,303–304
tests supporting, 303–304
类生产数据、自动化数据库和309–310
Production-like data, automating databases and, 309–310
职业发展,57
Professional development, 57
分析工具,234
Profiling tools, 234
程序员
Programmers
对自动化的态度,265–266
attitude regarding automation, 265–266
大图测试,397
big picture tests, 397
合作,413-414
collaboration with, 413–414
在迭代规划期间考虑所有观点,387–389
considering all viewpoints during iteration planning, 387–389
促进沟通,429-430
facilitating communication and, 429–430
使用400–401审查高级测试
reviewing high-level tests with, 400–401
测试人员与开发人员的比例,66–67
tester-developer ratio, 66–67
培训, 61
training, 61
写任务卡和,391
writing task cards and, 391
项目管理办公室 (PMO),440
Project Management Office (PMO), 440
项目,PAS 示例,176–177
Projects, PAS example, 176–177
原型
Prototypes
作为通用语言可访问,134
accessible as common language, 134
模型和160
mock-ups and, 160
纸, 22 , 138–139 , 380 , 400 , 414
paper, 22, 138–139, 380, 400, 414
纸与绿野仙踪类型,275
paper vs. Wizard of Oz type, 275
UI(用户界面),107
UI (user interface), 107
脉冲,126
Pulse, 126
Python 的 PyUnit 单元测试工具,126
PyUnit unit test tool for Python, 126
定义,504
definition, 504
在职称中,31
in job titles, 31
独立QA团队,60人
independent QA team, 60
可与“测试”互换,59
interchangeable with “test,” 59
整个团队的方法,39
whole team approach, 39
在传统团队中工作,9
working on traditional teams, 9
象限 1。参见 面向技术的测试,支持团队(象限 1)
Quadrant 1. See Technology-facing tests, supporting team (Quadrant 1)
象限 2。参见 面向业务的测试,支持团队(象限 2)
Quadrant 2. See Business-facing tests, supporting team (Quadrant 2)
第 3 象限。参见 面向业务的测试,评价产品(第 3 象限)
Quadrant 3. See Business-facing tests, critiquing the product (Quadrant 3)
第 4 象限。参见 面向技术的测试,评价产品(第 4 象限)
Quadrant 4. See Technology-facing tests, critiquing the product (Quadrant 4)
象限
Quadrants
自动化测试类别,274–276
automation test categories, 274–276
面向业务(Q2 和 Q3),97–98
business facing (Q2 & Q3), 97–98
上下文驱动的测试和,106-108
context-driven testing and, 106–108
批评产品(Q3 和 Q4),104
critiquing the product (Q3 & Q4), 104
管理技术债务,106
managing technical debt, 106
概述, 97–98
overview of, 97–98
作为规划指南,490
as planning guide, 490
测试的目的和,97
purpose of testing and, 97
第 1 象限摘要,99
Quadrant 1 summary, 99
第 2 象限摘要,99–100
Quadrant 2 summary, 99–100
第 3 象限摘要,101–102
Quadrant 3 summary, 101–102
象限 4 摘要,102–104
Quadrant 4 summary, 102–104
共同责任和105–106
shared responsibility and, 105–106
故事完成,104-105
story completion and, 104–105
支持团队(Q1 和 Q2),100–101
supporting the team (Q1 & Q2), 100–101
面向技术(Q1 和 Q4),97–98
technology facing (Q1 & Q4), 97–98
质量
Quality
客户在制定质量标准中的作用,26
customer role in setting quality standards, 26
型号,90–93
models, 90–93
organizational philosophy regarding, 38–40
质量保证。参见 QA(质量保证)
Quality assurance. See QA (quality assurance)
素质警察心态,57
Quality police mentality, 57
问题,引出需求,135-136
Questions, for eliciting requirements, 135–136
雷达图,XP,47–48
Radar charts, XP, 47–48
乔纳森·拉斯穆森,11 岁
Rasmusson, Jonathan, 11
录制/回放工具
Record/playback tools
automation strategy and, 294, 296–297
GUI 测试工具,171–172
GUI test tools, 171–172
恢复测试,459
Recovery testing, 459
冗余测试,232
Redundancy tests, 232
重构
Refactoring
定义,504
definition, 504
支持的 IDE,124–126
IDEs supporting, 124–126
回归套房,434
Regression suite, 434
回归测试,432–434
Regression tests, 432–434
作为安全网的自动回归测试,261–262
automated regression tests as a safety net, 261–262
自动化作为成功因素,484
automating as success factor, 484
检查大图,434
checking big picture, 434
定义,504
definition, 504
探索性测试和212
exploratory testing and, 212
保持构建“绿色”,433
keeping the build “green,” 433
保持快速构建,433–434
keeping the build quick, 433–434
记录错误和420
logging bugs and, 420
回归套房和,434
regression suite and, 434
发布候选人和458
release candidates and, 458
发布
Release
验收标准,470–473
acceptance criteria, 470–473
管理, 474
management, 474
产品交付,470
product delivery, 470
如果还没有准备好怎么办,463–464
what if it is not ready, 463–464
发布候选人
Release candidates
具有挑战性的候选发布版本,473
challenging release candidate builds, 473
定义,505
definition, 505
测试, 458
testing, 458
发布指标
Release metrics
代码覆盖率,360–364
code coverage, 360–364
缺陷指标,364–366
defect metrics, 364–366
概述, 358
overview of, 358
通过测试,358–360
passing tests, 358–360
发行说明,474
Release notes, 474
发布计划,329–367
Release planning, 329–367
概述, 329
overview of, 329
优先考虑,338–340
prioritizing and, 338–340
目的,330-331
purpose of, 330–331
范围,340–344
scope, 340–344
尺寸和332–337
sizing and, 332–337
测试计划备选方案,350–354
test plan alternatives, 350–354
测试计划,345–350
test planning, 345–350
能见度和354–366
visibility and, 354–366
可靠性测试
Reliability testing
概述,230-231
overview of, 230–231
远程数据监控系统示例,250–251
Remote Data Monitoring system example, 250–251
远程数据监控系统示例
Remote Data Monitoring system example
验收测试,245
acceptance tests, 245
应用程序,242–243
application, 242–243
应用测试象限,252–253
applying test quadrants, 252–253
自动化功能测试结构,245–247
automated functional test structure, 245–247
记录测试代码,251
documenting test code, 251
嵌入式测试,248
embedded testing, 248
端到端测试,249–250
end-to-end tests, 249–250
探索性测试,248–249
exploratory testing, 248–249
概述, 242
overview of, 242
可靠性测试,250–251
reliability testing, 250–251
报告测试结果,251
reporting test results, 251
团队和流程,243–244
team and process, 243–244
测试数据源,249
testing data feeds, 249
单元测试,244–245
unit tests, 244–245
用户验收测试,250
user acceptance testing, 250
网络服务,247-248
web services, 247–248
远程团队成员。查看 地理上分散的团队
Remote team member. See Geographically dispersed teams
重复性任务,自动化,284
Repetitive tasks, automating, 284
报告
Reports
文档和208–210
documentation and, 208–210
远程数据监控系统示例,251
Remote Data Monitoring system example, 251
存储库,124
Repository, 124
要求
Requirements
面向业务的测试寻址,130
business-facing tests addressing, 130
402的文档
documentation of, 402
收集有关需求的所有观点,374-375
gathering all viewpoints regarding requirements, 374–375
如何引出,135-140
how to elicit, 135–140
无功能,218–219
nonfunctional, 218–219
窘境,132–134
quandary, 132–134
引出示例和要求的工具,155-156
tools for eliciting examples and requirements, 155–156
资源
Resources
完成故事,381
completing stories and, 381
招聘敏捷测试员,67-69
hiring agile tester, 67–69
概述, 66
overview of, 66
测试人员与开发人员的比例,66–67
tester-developer ratio, 66–67
测试和434–435
testing and, 434–435
响应时间
Response time
原料药,411
API, 411
负载测试和234–235
load testing and, 234–235
可衡量的目标,76
measurable goals and, 76
网络服务和207
web services and, 207
回顾
Retrospectives
持续改进,28
continuous improvement and, 28
改进的想法,447-449
ideas for improvement, 447–449
迭代计划和,383
iteration planning and, 383
概述, 444–445
overview of, 444–445
流程改进,90
process improvement and, 90
“开始、停止和继续”练习,445–447
“start, stop, and continue” exercise, 445–447
投资回报。请参见 ROI(投资回报率)
Return on investment. See ROI (return on investment)
奖励、绩效和,70–71
Rewards, performance and, 70–71
富客户端单元测试工具,127
Rich-client unit testing tools, 127
瑞星,琳达,121–122
Rising, Linda, 121–122
风险
Risk
风险分析, 198 , 286 , 290 , 345–346
risk analysis, 198, 286, 290, 345–346
风险评估,407-409
risk assessment, 407–409
测试缓解,147-149
test mitigating, 147–149
罗杰斯, 保罗, 242 , 310 , 388 , 398
Rogers, Paul, 242, 310, 388, 398
自动化和, 264
automation and, 264
定义,505
definition, 505
精益测量和,75
lean measurement and, 75
指标和,78-79
metrics and, 78–79
说经理的语言,55
speaking managerís language, 55
角色、功能、商业价值模式,155
Role, function, business value pattern, 155
角色
Roles
冲突或多重角色,45
conflicting or multiple roles, 45
48-49岁之间的文化差异
cultural differences among, 48–49
客户团队,7
customer team, 7
开发团队,7–8
developer team, 7–8
互动,8
interaction of, 8
RPG单位, 118
RPGUnit, 118
Ruby 测试::单元, 170
Ruby Test::Unit, 170
Ruby 与 Watir
Ruby with Watir
功能测试,247
functional testing, 247
GUI 测试,285
GUI testing, 285
识别缺陷,212
identifying defects with, 212
用于自动化测试的关键字或操作词,182
keywords or actions words for automating tests, 182
概述,172-174
overview of, 172–174
测试自动化,186
test automation with, 186
红宝石模拟,127
RubyMock, 127
规则、错误管理和425
Rules, managing bugs and, 425
安全测试,232
Safety tests, 232
桑托斯,拉斐尔,448
Santos, Rafael, 448
满意条件。查看 满意条件
Satisfaction conditions. See Conditions of satisfaction
可扩展性测试,233–234
Scalability testing, 233–234
情景测试,192–193
Scenario testing, 192–193
流程图和,194-195
flow diagrams and, 194–195
概述,192-195
overview of, 192–195
肥皂剧测试,193
soap opera tests, 193
范围,340–344
Scope, 340–344
面向业务的测试定义,134
business-facing tests defining, 134
截止日期和时间表,以及340–341
deadlines and timelines and, 340–341
关注价值,341-342
focusing on value, 341–342
概述, 340
overview of, 340
全系统影响,342
system-wide impact, 342
测试计划和345
test plans and, 345
第三方参与,342-344
third-party involvement and, 342–344
脚本
Scripts
自动比较,283
automating comparisons, 283
作为自动化工具,297
as automation tools, 297
转换脚本,461
conversion scripts, 461
数据生成工具,305
data generation tools, 305
探索性测试和211–212
exploratory testing and, 211–212
敏捷
Scrum
远程数据监控系统示例,244
Remote Data Monitoring system example, 244
冲刺评论,444
sprint reviews, 444
ScrumMaster
ScrumMaster
流程改进方法,448–449
approach to process improvement, 448–449
调整故事大小,336–337
sizing stories and, 336–337
写任务卡和,391
writing task cards and, 391
SDD (story test-driven development)
识别变异,410
identifying variations, 410
概述,262–263
overview of, 262–263
测试先行的开发和,263
test-first development and, 263
测试 Web 服务和170
testing web services and, 170
安全测试
Security testing
攻击者由外向内的方法,225
outside-in approach of attackers, 225
概述,223-227
overview of, 223–227
所需的专业知识,220
specialized knowledge required for, 220
硒
Selenium
GUI 测试工具,174–175
GUI test tools, 174–175
实施自动化,316–318
implementing automation, 316–318
开源工具,163
open source tools, 163
test automation with, 186, 316
自组织
Self-organization
原则,29-30
principles, 29–30
自组织团队,69
self-organizing teams, 69
基于会话的测试,200–201
Session-based testing, 200–201
设置
Setup
自动化,284–285
automating, 284–285
探索性测试,211–212
exploratory testing, 211–212
共享资源
Shared resources
访问, 43
access to, 43
专家作为,301
specialists as, 301
写作任务和,390
writing tasks and, 390
共同责任,105–106
Shared responsibility, 105–106
Shout-Out 鞋盒,450
Shout-Out Shoebox, 450
“给我看”,与程序员合作,413–414
“Show me,” collaboration with programmers, 413–414
简单
Simplicity
自动化和298–299
automation and, 298–299
编码,406
coding, 406
记录错误和428–429
logging bugs and, 428–429
“保持简单”的原则,26-27
principle of “keeping it simple,” 26–27
模拟器工具
Simulator tools
嵌入式测试和,248
embedded testing and, 248
概述, 213
overview of, 213
规模,组织性,42-43
Size, organizational, 42–43
上浆故事,332–337
Sizing stories, 332–337
例如,334–337
example of, 334–337
如何,332–333
how to, 332–333
概述, 332
overview of, 332
测试人员在333–334中的角色
tester’s role in, 333–334
适应性和,39-40
adaptability and, 39–40
与态度,20
vs. attitude, 20
持续改进原则,27
continuous improvement principle, 27
谁执行测试,以及220–221
who performs tests and, 220–221
小块,增量开发,144-146
Small chunks, incremental development, 144–146
肥皂
SOAP
定义,505
definition, 505
performance tests and, 223, 234
肥皂剧测试,193
Soap opera tests, 193
肥皂用户界面
soapUI
定义,505
definition, 505
performance tests and, 223, 234
测试 Web 服务,170–171
testing Web Services, 170–171
SOA测试,234
SOATest, 234
基于软件的工具,163
Software-based tools, 163
软件配置管理模式:有效的团队合作、实用的集成(Berczuk 和 Appleton),124
Software Configuration Management Patterns: Effective Teamwork, Practical Integrations (Berczuk and Appleton), 124
软件残局(盖伦),471
Software Endgames (Galen), 471
源代码控制
Source code control
的好处, 255
benefits of, 255
概述,123-124
overview of, 123–124
SOX 合规性,469
SOX compliance, 469
用同一个声音说话,客户,373-374
Speak with one voice, customers, 373–374
专业化,220-221
Specialization, 220–221
以速度为目标,112
Speed as a goal, 112
峰值、开发和测试,381
Spikes, development and test, 381
电子表格
Spreadsheets
测试电子表格,353
test spreadsheets, 353
引出示例和要求的工具,159
tools for eliciting examples and requirements, 159
Sprint reviews, 444. See also Demos/demonstrations
SQL* 加载程序,460
SQL*Loader, 460
稳定性测试,28
Stability testing, 28
暂存环境,458
Staging environment, 458
Stand-up meetings, 177, 429, 462
标准
Standards
可维护性和,227
maintainability and, 227
质量模型和,90-93
quality models and, 90–93
“开始、停止、继续”练习,回顾,445-447
“Start, stop, continue” exercise, retrospectives, 445–447
静态分析,安全测试工具,225
Static analysis, security testing tools, 225
钢螺纹,增量开发,144、338、345
Steel thread, incremental development, 144, 338, 345
故事。另请参阅 面向业务的测试
Stories. See also Business-facing tests
在迭代之前工作的好处,370–372
benefits of working on in advance of iterations, 370–372
129–130的简要说明
briefness of, 129–130
面向业务的测试,130
business-facing tests as, 130
确定故事大小,375–376
determining story size, 375–376
编码时专注于一个故事,411-412
focusing on one story when coding, 411–412
识别变异,410
identifying variations, 410
知道故事何时结束,104–105
knowing when a story is done, 104–105
记录错误和420–421
logging bugs and, 420–421
模型和380
mock-ups and, 380
优先排序,338–340
prioritizing, 338–340
资源和,381
resources and, 381
范围和,340
scope and, 340
浆纱。查看 上浆故事
sizing. See Sizing stories
定义的故事测试,505
story tests defined, 505
全系统影响,342
system-wide impact of, 342
测试计划和345
test plans and, 345
测试策略和,380–381
test strategies and, 380–381
可测试,393–396
testable, 393–396
将错误视为425
treating bugs as, 425
燃尽图,429
burndown charts, 429
定义,505–506
definition, 505–506
示例,356–357
examples, 356–357
物理, 356
physical, 356
贴纸和,355
stickers and, 355
正在进行的工作,390
work in progress, 390
故事卡
Story cards
审计和, 89
audits and, 89
处理错误和424–425
dealing with bugs and, 424–425
迭代计划和,244
iteration planning and, 244
故事叙述,409
story narrative on, 409
故事测试驱动开发。参见 SDD(故事测试驱动开发)
Story test-driven development. See SDD (story test-driven development)
Strangler 应用程序 (Fowler),116–117
Strangler application (Fowler), 116–117
战略
Strategy
自动化。请参阅 自动化策略
automation. See Automation strategy
测试计划与测试策略,86-87
test planning vs. test strategy, 86–87
测试策略,380–381
test strategies, 380–381
策略,用于编写测试
Strategy, for writing tests
逐步构建测试,178–179
building tests incrementally, 178–179
迭代计划和,372
iteration planning and, 372
保持测试通过,179
keep the tests passing, 179
概述,177-178
overview of, 177–178
测试设计模式,179–183
test design patterns, 179–183
可测性和,183–185
testability and, 183–185
压力测试。请参见 负载测试
Stress testing. See Load testing
成功因素。查看 关键成功因素
Success factors. See Key success factors
成功,庆祝
Successes, celebrating
改变实施,50-52
change implementation and, 50–52
迭代总结和,449-451
iteration wrap up and, 449–451
Sustainable pace, of testing, 40–41, 303
SWTBot GUI测试工具,127
SWTBot GUI test tool, 127
实践之间的协同作用,489
Synergy, between practices, 489
系统,故事的全系统影响,342
System, system-wide impact of story, 342
tail-f, 212
tail-f, 212
塔尔塔利亚、科尼、439、454、470、473 _ _ _
Tartaglia, Coni, 439, 454, 470, 473
任务板。查看 故事板
Task boards. See Story boards
任务卡
Task cards
自动化测试和394–395
automating testing and, 394–395
迭代计划和389–392
iteration planning and, 389–392
产品交付和462–463
product delivery and, 462–463
任务
Tasks
完成测试任务,415–416
completing testing tasks, 415–416
定义,505–506
definition, 505–506
自动驾驶测试,262–263
automated tests driving, 262–263
缺陷和490
defects and, 490
定义,506
definition, 506
概述,5
overview of, 5
测试优先开发与113–114相比
Test-First Development compared with, 113–114
团队城市,126
Team City, 126
团队结构,59–65
Team structure, 59–65
敏捷项目团队,64–65
agile project teams, 64–65
独立QA团队,60人
independent QA team, 60
将测试人员集成到敏捷项目中,61–63
integration of testers into agile project, 61–63
概述, 59
overview of, 59
传统职能结构与敏捷结构,64
traditional functional structure vs agile structure, 64
自动化作为团队的努力,484
automation as team effort, 484
建筑,69–71
building, 69–71
庆祝成功,50–52
celebrating success, 50–52
同地办公,65–66 岁
co-located, 65–66
控制工作量和393
controlling workload and, 393
顾客,7
customer, 7
开发人员,7–8
developer, 7–8
赋权,44
empowerment of, 44
促进沟通,429-432
facilitating communication and, 429–432
geographically dispersed, 376–378, 431–432
给予所有团队成员同等的权重,31
giving all team members equal weight, 31
给予所有权,50
giving ownership to, 50
hiring agile tester for, 67–69
客户和开发团队之间的互动,8
interaction between customer and developer teams, 8
迭代计划和384–385
iteration planning and, 384–385
物流, 59
logistics, 59
解决问题和,123
problem solving and, 123
远程数据监控系统示例,243–244
Remote Data Monitoring system example, 243–244
共同责任和105–106
shared responsibility and, 105–106
传统的,9-10
traditional, 9–10
using tests to support Quadrants 1 and 2, 100–101
整个团队的方法。查看 整个团队方法
whole team approach. See Whole team approach
在敏捷团队中工作,10-12
working on agile teams, 10–12
拆解,用于测试,307–308
Teardown, for tests, 307–308
技术债务
Technical debt
缺陷,418
defects as, 418
定义,506
definition, 506
概述,5
overview of, 5
象限 1 & 4, 97–98
Quadrants 1 & 4, 97–98
面向技术的测试,批评产品(第 4 象限),217–239
Technology-facing tests, critiquing the product (Quadrant 4), 217–239
基线,235–237
baselines, 235–237
编码和测试,以及412–413
coding and testing and, 412–413
兼容性测试,229–230
compatibility testing, 229–230
安装性测试,231-232
installability testing, 231–232
互操作性测试,228–229
interoperability testing, 228–229
可维护性测试,227-228
maintainability testing, 227–228
内存管理测试,237–238
memory management testing, 237–238
概述,217-219
overview of, 217–219
性能和负载测试,234
performance and load testing, 234
性能和负载测试工具,234-235
performance and load testing tools, 234–235
reliability testing, 230–231, 250–251
可扩展性测试,233–234
scalability testing, 233–234
安全测试,223-227
security testing, 223–227
测试环境和,237
test environment and, 237
何时使用,222–223
when to use, 222–223
谁执行测试,220–222
who performs the test, 220–222
面向技术的测试,支持团队(象限 1)
Technology-facing tests, supporting team (Quadrant 1)
构建工具,126
build tools, 126
设计时考虑到测试,115–118
designing with testing in mind, 115–118
轻松完成任务,114–115
ease of accomplishing tasks, 114–115
IDE,124–126
IDEs for, 124–126
基础设施支持,111-112
infrastructure supporting, 111–112
概述,109-110
overview of, 109–110
目的,110–111
purpose of, 110–111
源代码控制,123–124
source code control, 123–124
速度的好处,112–114
speed as benefit of, 112–114
及时反馈,118-119
timely feedback, 118–119
工具包,123
toolkit for, 123
单元测试工具,126–127
unit test tools, 126–127
单元测试,244–245
unit tests, 244–245
如果团队不执行这些测试怎么办,121–123
what to do if team doesn’t perform these tests, 121–123
在哪里/何时停止,119–121
where/when to stop, 119–121
测试自动化金字塔
Test automation pyramid
多层自动化方法,290-291
multi-layered approach to automation and, 290–291
概述,276–279
overview of, 276–279
三只小猪的比喻,278
three little pigs metaphor, 278
UI 背后的测试,282
Test behind UI, 282
测试用例
Test cases
增加复杂性,407
adding complexity, 407
作为文档,402
as documentation, 402
示例驱动开发,379
example-driven development, 379
识别变异,410
identifying variations, 410
从简单开始,406
starting simple, 406
测试覆盖率(和/或代码覆盖率),360–364
Test coverage (and/or code coverage), 360–364
测试设计模式,179–183
Test design patterns, 179–183
构建/操作/检查模式,180
Build/Operate/Check pattern, 180
数据驱动和关键字驱动的测试,182–183
data-driven and keyword-driven tests, 182–183
概述, 179
overview of, 179
测试起源模式(Veragen),179
test genesis patterns (Veragen), 179
基于时间的活动和事件模式,181
time-based, activity, and event patterns, 181
测试双打
Test doubles
定义,506
definition, 506
分层架构和,116
layered architectures and, 116
测试驱动开发。请参见 TDD(测试驱动开发)
Test-driven development. See TDD (test-driven development)
测试先行开发
Test-First Development
定义,506
definition, 506
与 TDD(测试驱动开发)相比,113–114
TDD (test-driven development) compared with, 113–114
测试管理,186
Test management, 186
测试管理工具包(象限 2),186
Test management toolkit (Quadrant 2), 186
测试计划备选方案,350 -354
Test plan alternatives, 350-354
测试计划,345–350
Test planning, 345–350
自动化测试列表,测试计划备选方案,353–354
automated test lists, test plan alternatives, 353–354
基础设施和346–347
infrastructure and, 346–347
写作的原因,345-346
reasons for writing, 345–346
测试环境和,347–348
test environment and, 347–348
测试计划备选方案,350–354
test plan alternatives, 350–354
测试计划,轻量级350
test plans, lightweight 350
测试计划样本,351
test plan sample, 351
测试策略对比,86-88
test strategy vs., 86–88
可追溯性和88
traceability and, 88
测试类型和,346
types of tests and, 346
从哪里开始,345
where to start, 345
检测结果
Test results
沟通,357–358
communicating, 357–358
组织,322–324
organizing, 322–324
发布计划和349–350
release planning and, 349–350
测试技能。查看 技能
Test skills. See Skills
测试钉,381
Test spikes, 381
测试电子表格,353
Test spreadsheets, 353
测试策略
Test strategy
迭代、预迭代活动和380–381
iterations, pre-iteration activities and, 380–381
测试计划对比,86-88
test plan vs., 86–88
测试存根
Test stubs
定义,506
definition, 506
与外部应用程序集成,以及459
integration with external applications and, 459
单元测试和,127
unit tests and, 127
Test teams, 506–507. See also Teams
测试工具。另见 工具包
Test tools. See also Toolkits
API 层功能,168–170
API-layer functional, 168–170
探索性测试,210–211
exploratory testing, 210–211
生成测试数据,212
generating test data with, 212
GUI 测试,170–176
GUI tests, 170–176
自制, 175
home-brewed, 175
自产, 314
home-grown, 314
IDE,124–126
IDEs, 124–126
性能测试,234–235
performance testing, 234–235
安全测试,225
security testing, 225
单元级测试,126-127,165-168
unit-level tests, 126–127, 165–168
网络服务测试,170
web service tests, 170
阿尔法/贝塔,466–467
alpha/beta, 466–467
探索性的。参见 探索性测试 (ET)
exploratory. See Exploratory testing (ET)
功能性的。请参见 功能测试
functional. See Functional testing
图形用户界面。请参见 GUI 测试
GUI. See GUI testing
加载。请参见 负载测试
load. See Load testing
表现。请参见 性能和负载测试
performance. See Performance and load testing
压力。请参见 负载测试
stress. See Load testing
单元。请参见 单元测试
unit. See Unit testing
可用性。参见 可用性测试
usability. See Usability testing
用户验收测试。参见 UAT(用户验收测试)
user acceptance testing. See UAT (user acceptance testing)
测试写作策略。参见 Strategy,用于编写测试
Test writing strategy. See Strategy, for writing tests
可测试性,183–185
Testability, 183–185
自动与手动象限 2 测试,185
automated vs. manual Quadrant 2 tests, 185
自动化和149–150
automation and, 149–150
代码设计和测试设计,184–185
code design and test design and, 184–185
概述, 183
overview of, 183
故事数,393–396
of stories, 393–396
增加价值,12
adding value, 12
敏捷测试心态,20-21
agile testing mindset, 20–21
自动化允许专注于更重要的工作,260
automation allowing focus on more important work, 260
与客户合作,396–397
collaboration with customers, 396–397
在迭代计划中考虑所有观点,386–389
considering all viewpoints during iteration planning, 386–389
控制工作量和393
controlling workload and, 393
定义,507
definition, 507
促进沟通,429-430
facilitating communication, 429–430
反馈和,486
feedback and, 486
招聘敏捷测试员,67-69
hiring agile tester, 67–69
如何影响测试,121–122
how to influence testing, 121–122
将测试人员集成到敏捷项目中,61–63
integration of testers into agile project, 61–63
迭代和,327
iterations and, 327
让工作更轻松,114–115
making job easier, 114–115
上浆故事,333–334
sizing stories, 333–334
测试人员与开发人员的比例,66–67
tester-developer ratio, 66–67
写任务卡和,391
writing task cards and, 391
测试员的权利法案,49–50
Tester's bill of rights, 49–50
测试
Testing
同时编码和测试,409–410
coding and testing simultaneously, 409–410
完成测试任务,415–416
completing testing tasks, 415–416
识别变异,410
identifying variations, 410
管理,320-322
managing, 320–322
组织测试结果,322–324
organizing test results, 322–324
组织测试,319–322
organizing tests, 319–322
计划时间,455–456
planning time for, 455–456
开发后周期,467–468
post-development cycles, 467–468
象限。见象限
quadrants. See Quadrants
发布候选人,458
release candidates, 458
风险评估和407–409
risk assessment and, 407–409
可持续步伐,40-41
sustainable pace of, 40–41
传统与敏捷,12–15
traditional vs. agile, 12–15
测试的透明度,321-322
transparency of tests, 321–322
在上下文中测试
Testing in context
上下文驱动的测试和,106-108
context-driven testing and, 106–108
定义,502
definition, 502
TestNG GUI测试工具,127
TestNG GUI test tool, 127
永不失败的测试,286
Tests that never fail, 286
文本矩阵,350–353
Text matrices, 350–353
研磨机,234
The Grinder, 234
主题。另请参阅 发布计划
Themes. See also Release planning
定义,507
definition, 507
优先报道,339
prioritizing stories and, 339
写任务卡和,392
writing task cards and, 392
薄片、增量开发和338
Thin slices, incremental development and, 338
第三方
Third parties
兼容性测试和230
compatibility testing and, 230
发布计划和342–344
release planning and, 342–344
软件, 163
software, 163
托尔森,迈克,203
Tholfsen, Mike, 203
三只小猪的比喻,278
Three little pigs metaphor, 278
时间表、范围和340–341
Timelines, scope and, 340–341
工具包(象限 1)
Toolkit (Quadrant 1)
构建工具,126
build tools, 126
IDE,124–126
IDEs, 124–126
概述, 123
overview of, 123
源代码控制,123–124
source code control, 123–124
单元测试工具,126–127
unit test tools, 126–127
API 层功能测试工具,168–170
API-layer functional test tools, 168–170
自动化工具,164–165
automation tools, 164–165
逐步构建测试,178–179
building tests incrementally, 178–179
清单,156
checklists, 156
流程图,160-163
flow diagrams, 160–163
GUI 测试工具,170–176
GUI test tools, 170–176
保持测试通过,179
keep the tests passing, 179
思维导图,156–158
mind maps, 156–158
模型,160
mock-ups, 160
基于软件的工具,163
software-based tools, 163
电子表格,159
spreadsheets, 159
写作测试的策略,177-178
strategies for writing tests, 177–178
测试设计模式,179–183
test design patterns, 179–183
测试管理,186
test management, 186
可测性和,183–185
testability and, 183–185
工具策略,153–155
tool strategy, 153–155
引出示例和要求的工具,155-156
tools for eliciting examples and requirements, 155–156
单元级测试工具,165–168
unit-level test tools, 165–168
Web服务测试工具,170
Web service test tool, 170
工具包(第 3 象限)
Toolkit (Quadrant 3)
仿真器工具,213–214
emulator tools, 213–214
监控工具,212-213
monitoring tools, 212–213
模拟器工具,213
simulator tools, 213
用户验收测试,250
user acceptance testing, 250
工具包(第 4 象限)
Toolkit (Quadrant 4)
基线,235–237
baselines, 235–237
性能和负载测试工具,234-235
performance and load testing tools, 234–235
工具
Tools
API 层功能测试工具,168–170
API-layer functional test tools, 168–170
自动化,164–165
automation, 164–165
数据生成,304–305
data generation, 304–305
缺陷跟踪,83–85
defect tracking, 83–85
引出示例和要求,155-156,159-163
eliciting examples and requirements, 155–156, 159–163
仿真器工具,213–214
emulator tools, 213–214
探索性测试,210–211
exploratory testing, 210–211
生成测试数据,212
generating test data, 212
GUI 测试工具,170–176
GUI test tools, 170–176
自制, 175
home-brewed, 175
自产, 314
home-grown, 314
IDE,124–126
IDEs, 124–126
负载测试,234–235
load testing, 234–235
监测,212-213
monitoring, 212–213
性能测试,234–235
performance testing, 234–235
对于产品所有者和业务专家,134
for product owners and business experts, 134
安全测试,225
security testing, 225
模拟器, 213
simulators, 213
基于软件,163
software-based, 163
单元级测试,126-127,165-168
unit-level tests, 126–127, 165–168
供应商/商业,315-316
vendor/commercial, 315–316
Web服务测试工具,170
web service test tool, 170
工具、自动化
Tools, automation
敏捷友好型,316
agile-friendly, 316
一次使用一个工具,312–313
applying one tool at a time, 312–313
自制, 175
home-brewed, 175
自产, 314
home-grown, 314
确定工具要求,311–312
identifying tool requirements, 311–312
开源,314–315
open source, 314–315
选择,294–298
selecting, 294–298
供应商,315–316
vendors, 315–316
可追溯性
Traceability
DTS 和82
DTS and, 82
矩阵,86
matrices, 86
测试计划和,88
test planning and, 88
跟踪、测试任务和状态,354–357
Tracking, test tasks and status, 354–357
传统流程,转型。请参见将传统流程转变为敏捷流程
Traditional processes, transitioning. See Transitioning traditional processes to agile
传统球队,9–10
Traditional teams, 9–10
传统与敏捷测试,12–15
Traditional vs. agile testing, 12–15
训练
Training
作为可交付成果,469
as deliverable, 469
缺乏, 45
lack of, 45
将传统流程转变为敏捷流程,73–93
Transitioning traditional processes to agile, 73–93
缺陷跟踪。请参见 缺陷跟踪
defect tracking. See Defect tracking
现有流程和,88–92
existing process and, 88–92
精益测量,74-75
lean measurements, 74–75
轻量级流程和,73–74
lightweight processes and, 73–74
指标和,74-79
metrics and, 74–79
概述, 73
overview of, 73
测试计划。请参见 测试计划
test planning. See Test planning
UAT(用户验收测试)
UAT (user acceptance testing)
开发后测试周期,467–468
post-development testing cycles, 467–468
产品交付和464–466
product delivery and, 464–466
在象限 3, 102
in Quadrant 3, 102
release planning for, 331, 346
远程数据监控系统示例,250
Remote Data Monitoring system example, 250
在测试计划中,351
in test plan, 351
尝试新功能,102
tryng out new features and, 102
在迭代启动会议上写作,372
writing at iteration kickoff meeting, 372
UI(用户界面)。另见GUI(图形用户界面)
UI (user interface). See also GUI (graphical user interface)
自动化策略和,293
automation strategy and, 293
建模和, 399
modeling and, 399
单元测试工具,165–168。另请参阅单个单元工具
Unit test tools, 165–168. See also by individual unit tools
行为驱动的开发工具,166–168
behavior-driven development tools, 166–168
列表,126-127
list of, 126–127
概述, 165
overview of, 165
自动化, 282
automating, 282
BDD(行为驱动开发),165–168
BDD (Behavior-driven development), 165–168
定义,507
definition, 507
指标和,76
metrics and, 76
支持功能,5
supporting function of, 5
TDD(测试驱动开发)和,111
TDD (test-driven development) and, 111
面向技术的测试,120
technology-facing tests, 120
象限 1 测试的工具,126–127
tools for Quadrant 1 tests, 126–127
可用性测试,202–204
Usability testing, 202–204
检查竞争对手的申请,204
checking out applications of competitors, 204
导航和,204
navigation and, 204
概述, 202
overview of, 202
用户需求和角色测试,202-204
users needs and persona testing, 202–204
什么不应该自动化,285–286
what should not be automated, 285–286
用例,398
Use cases, 398
用户验收测试。参见 UAT(用户验收测试)
User acceptance testing. See UAT (user acceptance testing)
用户文档,207–208
User documentation, 207–208
用户界面 (UI)。另见GUI(图形用户界面)
User interface (UI). See also GUI (graphical user interface)
自动化策略和,293
automation strategy and, 293
建模和, 399
modeling and, 399
用户故事。看故事
User story. See Story
用户故事卡。查看故事卡
User story card. See Story card
应用于敏捷软件开发的用户故事(Cohn),155
User Stories Applied for Agile Software Development (Cohn), 155
Vaage,卡罗尔,330
Vaage, Carol, 330
价值
Value
添加,31-33
adding, 31–33
交付给客户,22-23
delivering to customer, 22–23
专注于,341–342
focusing on, 341–342
测试人员添加,12
testers adding, 12
价值观,敏捷,3–4。另请参阅原则,适用于敏捷测试人员
Values, agile, 3–4. See also Principles, for agile testers
变体、编码和测试以及410
Variations, coding and testing and, 410
速度
Velocity
职业倦怠率和,79
burnout rate and, 79
数据库影响,228
database impact on, 228
缺陷和,487
defects and, 487
定义,507
definition, 507
最大化,370
maximizing, 370
可持续的测试速度,41
sustainable pace of testing and, 41
花时间做正确的事,301
taking time to do it right, 301
technical debt and, 106, 313, 418, 506
供应商
Vendors
自动化工具,315–316
automation tools, 315–316
捕捉回放工具,267
capture-playback tool, 267
集成开发环境,125
IDEs, 125
规划和342–344
planning and, 342–344
源代码控制工具,124
source code control tools, 124
维拉根,皮埃尔,76 岁,163 岁,179 岁,295 岁,363 岁,372 岁,444 岁
Veragen, Pierre, 76, 163, 179, 295, 363, 372, 444
版本控制,123–124,186。另请参阅源代码控制
Version control, 123–124, 186. See also Source Code Control
观点。另见大图
Viewpoints. See also Big picture
在迭代规划期间考虑所有观点,385–389
considering all viewpoints during iteration planning, 385–389
收集有关需求的所有观点,374-375
gathering all viewpoints regarding requirements, 374–375
三的幂和,411
Power of Three and, 411
在引出需求时使用多种观点,137–138
using multiple viewpoints in eliciting requirement, 137–138
能见度,354–366
Visibility, 354–366
代码覆盖率,360–364
code coverage, 360–364
交流测试结果,357–358
communicating test results, 357–358
缺陷指标,364–366
defect metrics, 364–366
通过测试的次数,358–360
number of passing tests, 358–360
概述, 354
overview of, 354
发布指标,358
release metrics, 358
跟踪测试任务和状态,354–357
tracking test tasks and status, 354–357
视觉工作室,125
Visual Studio, 125
约翰·沃里斯,117 岁
Voris, John, 117
瀑布方法,开发
Waterfall approach, to development
与敏捷开发相比,12–13
agile development compared with, 12–13
“迷你瀑布”现象,46-47
ìmini-waterfallî phenomenon, 46–47
成功,112
successes of, 112
测试计划和346
test plans and, 346
Watir(Ruby中的Web应用程序测试),163、172–174、320。另请参阅Ruby 与 Watir
Watir (Web Application Testing in Ruby), 163, 172–174, 320. See also Ruby with Watir
Web 服务描述语言 (WSDL),507
Web Services Description Language (WSDL), 507
网络服务测试
Web service testing
自动化, 282
automating, 282
概述, 207
overview of, 207
远程数据监控系统示例,247–248
Remote Data Monitoring system example, 247–248
工具,170–171
tools for, 170–171
网络负载,234
WebLoad, 234
惠兰,德克兰,321
Whelan, Declan, 321
白板
Whiteboards
示例驱动开发,379
example-driven development, 379
促进沟通,430
facilitating communication, 430
建模, 399
modeling, 399
规划图,371
planning diagram, 371
与程序员一起审查高级测试,400–401
reviewing high-level tests with programmers, 400–401
测试计划备选方案,353–354
test plan alternatives, 353–354
整个团队方法,325
Whole team approach, 325
的优点, 26
advantages of, 26
敏捷与传统开发,15–16
agile vs. traditional development, 15–16
自动化策略和,300-301
automation strategy and, 300–301
预算限制和55
budget limits and, 55
在工作中找到乐趣,31
finding enjoyment in work and, 31
将测试人员与程序员配对,279
pairing testers with programmers, 279
共同责任和105–106
shared responsibility and, 105–106
团队建设,69
team building and, 69
团队结构和,59–62
team structure and, 59–62
测试自动化,270
to test automation, 270
测试管理和,322
test management and, 322
与传统跨职能团队相比,64
traditional cross-functional team compared with, 64
团队成员的价值和,70
value of team members and, 70
维基百科
Wiki
作为通讯工具,164
as communication tool, 164
示例的图形文档,398–399
graphical documentation of examples, 398–399
要求,402
requirements, 402
故事清单和,156
story checklists and, 156
测试用例,372
test cases, 372
可追溯性和88
traceability and, 88
威尔逊 - 威尔士,帕特里克,278
Wilson-Welsh, Patrick, 278
绿野仙踪测试,138-139
Wizard of Oz Testing, 138–139
工作流程图,398
Workflow diagrams, 398
Working Effectively With Legacy Code (Feathers), 117, 288
工作量,393
Workload, 393
Worst-case scenarios, 136, 334
编写测试,策略。参见Strategy,用于编写测试
Writing tests, strategy for. See Strategy, for writing tests
WSDL(网络服务描述语言),507
WSDL (Web Services Description Language), 507
敏捷团队拥抱,10-11
agile team embracing, 10–11
勇气作为核心价值,25
courage as core value in, 25
xUnit, 126–127
xUnit, 126–127
亚基奇,乔,316
Yakich, Joe, 316
零错误容忍,79,418–419